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ABSTRACT 


Femtocells  are  low  power  base  stations  that  communicate  through  a  licensed  spectrum 
with  the  intent  to  improve  coverage  and  perfonnance  of  voice  and  broadband  services. 
The  Femtocell  works  through  a  cellular  network  provider  to  enhance  cellular 
portable/mobile  devices  especially  in  locations  where  coverage  by  cellular  systems  using 
large  cells  is  weak  and  intermittent. 

The  use  of  smartphones,  tablets,  and  other  wireless  devices  is  becoming 
increasingly  prevalent  and  is  driving  the  need  for  innovations  in  wireless  data 
technologies  to  provide  more  capacity,  higher  speed  connections,  and  higher  quality  of 
service.  Femtocells  can  provide  a  useful  way  for  mobile  operators  to  offer  a  better  user 
experience  and  deliver  broadband  services  indoors  consistently  and  reliably  for  a 
comparable  context  of  application,  distances,  and  obstacles. 

In  this  thesis  we  will  conduct  a  quantitative  and  qualitative  analysis  of  Femtocell 
performance  in  comparison  to  that  of  Wi-Fi.  Using  COTS  Femtocell  and  Wi-Fi 
technology  an  analysis  will  be  conducted  to  establish  which  of  the  two  is  the  better  means 
of  bringing  internet  connectivity  to  forward  deployed  forces.  The  potential  benefits  of 
this  research  are  a  better  understanding  of  the  advantages  and  disadvantages  of  Femtocell 
and  Wi-Fi  networks  in  simulated  garrison  and  deployed  environments. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Femtocells  are  known  by  most  as  a  type  of  home  cellular  base-station. 
Commercially,  it  has  names  like  Network  Extender  (Verizon),  Microcells  (AT&T),  and 
Airave  (Sprint),  among  others,  worldwide.  It  is  a  low-power  base  station  that 
communicates  through  a  licensed  spectrum  with  the  intent  to  improve  indoor  coverage 
and  perfonnance  of  voice  and  broadband  services.  The  femtocell,  working  through  a 
cellular  network  provider,  enhances  connectivity  for  cellular  phones,  smartphones,  and 
other  portable/mobile  devices,  especially  in  locations  where  coverage  by  cellular  systems 
using  large  cells  is  weak  and  intermittent  (i.e.,  indoors  or  remote  areas).  Ultimately,  the 
user’s  mobile  devices  are  connected  via  the  femtocell  to  a  backbone  network  supplied  by 
an  Internet  service  provider. 

Although  the  femtocell  architecture  may  seem  different  to  the  casual  user,  it  is 
actually  the  same  connection  that  a  normal  mobile  device  uses  to  access  Internet 
connectivity.  The  difference  is  that  a  standard  device  connects  through  outdoor  high 
power  base  stations  and  the  femtocell  is  its  own  access  point  base  station.  This  smaller 
localized  base  station  provides  very  stable  and  efficient  Internet  connections. 

Femtocell  network  technology  may  seem  like  a  new  technology  but  early 
femtocell  research  was  introduced  in  the  late  1990s  and  has  grown  dramatically  in  the  last 
decade.  Both  the  general  public  and  commercial  mobile  operators  have  shown  increased 
interest  in  ways  to  improve  upon  and  expand  this  technology  through  3G  and  LTE/4G. 

Questions  often  asked  are,  “Why  do  we  need  femtocells  when  there  is  Wi-Fi 
technology,”  and  “Which  one  is  better  for  the  potential  mobile  device  user?”  The 
purpose  of  this  thesis  is  to  analyze  femtocell  and  Wi-Fi  network  capabilities  and 
performance  to  detennine  which  is  the  better  platfonn  for  military  use  in  a  potential 
tactical  network. 
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B.  PURPOSE 

The  purpose  of  this  thesis  is  to  conduct  an  analysis  of  femtocell  performance  in 
comparison  to  that  of  Wi-Fi  from  the  user’s  point  of  view.  Using  commercial-off-the- 
shelf  (COTS)  femtocell  and  Wi-Fi  technology,  an  analysis  will  be  conducted  to  establish 
which  of  the  two  is  the  better  means  of  bringing  Internet  connectivity  to  forward 
deployed  military  members.  In  order  to  accomplish  this  we  will  perform  perfonnance 
tests  in  the  areas  of  Internet  connectivity,  uploading  and  downloading  speeds,  and  Voice 
over  IP  (VoIP)  in  both  ideal  conditions  and  in  realistic  (less  than  ideal)  conditions.  The 
potential  benefits  of  this  research  to  the  defense  establishment  are  a  better  understanding 
of  the  advantages  and  disadvantages  of  femtocell  and  Wi-Fi  networks  in  simulated 
garrison  and  deployed  environments. 

C.  SCOPE 

Our  objective  for  this  research  is  a  better  understanding  and  analysis  of  femtocell 
network  performance,  especially  as  compared  to  those  of  Wi-Fi.  Our  analysis  will  be 
based  on  several  perfonnance  tests  between  a  femtocell  and  a  Wi-Fi  in  the  areas  of 
accessing  the  Internet,  streaming  data,  and  voice  over  IP  (VoIP). 

In  order  to  perform  these  tests  we  utilize  a  COTS  Wi-Fi  router  and  femtocell.  The 
first  series  of  tests  will  be  baseline  performance  tests  that  will  establish  the  basic 
performances  in  ideal  conditions.  We  will  then  perform  the  same  series  of  tests  in  a 
realistic  environment  (non-ideal  situations). 

As  mentioned  earlier,  the  targeted  areas  of  testing  are  accessing  the  Internet, 
streaming  data,  and  VoIP.  These  areas  were  chosen  because  they  are  the  most  used 
methods  of  utilizing  networks.  Each  of  these  areas  can  also  be  used  by  members  of  the 
military  in  both  garrison  and  deployed  environments. 

The  first  area  is  accessing  the  Internet  through  Hyper  Text  Transfer  Protocol 
(HTTP).  HTTP  is  the  underlying  protocol  that  is  used  by  the  World  Wide  Web.  It 
defines  the  way  in  which  messages  are  formatted  and  transmitted,  as  well  as,  what  actions 
need  to  be  taken.  HTTP  is  used  in  every  action  of  the  process  of  accessing  the  web. 
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Real-time  Transport  Protocol  (RTP)  is  the  underlying  protocol  in  our  next  area  of 
evaluation:  streaming  data.  RTP  basically  standardizes  packet  fonnats  for  delivering 
audio  and  video  over  IP  networks.  RPT  is  used  for  streaming  media,  teleconferencing, 
and  real  time  data. 

The  final  area  of  testing  is  VoIP.  VoIP  is  the  process  of  transmitting  voice  traffic 
over  IP-based  networks.  VoIP  essentially  compresses  data  packets  during  transmission 
which  allows  more  data  to  be  handled  over  the  carrier.  As  a  result  VoIP  can  not  only 
handle  multiple  callers  at  once,  but  it  can  also  (through  software  applications)  transmit 
video  and  data. 

As  stated,  the  testing  will  be  conducted  in  both  ideal  and  non-ideal  environments. 
Our  definition  of  an  ideal  situation  is  one  where  we  are  located  in  the  same  room  as  the 
Wi-Fi  router  and  Femtocell  access  point.  The  non-ideal  environments  consist  of  moving 
further  away  from  the  router  and  Femtocell,  and  include  obstacles  such  as  walls  and 
floors.  The  number  of  users  on  the  network  is  also  included  in  our  non-ideal 
environments. 

D.  LITERATURE  REVIEW 

Femtocells  first  came  to  light  in  1996  through  Silventoinen  et  al.’s,  “Analysis  of  a 
new  channel  access  method  for  home  base-stations.”  It  described  the  potential  of 
extending  the  concept  of  a  home  base-station.  He  described  a  simple  architecture  of  a 
cellular  network  that  used  a  peculiar  Total  Frequency  Hopping.  This  basic  idea  led  to 
suggestions  of  a  requirement  to  double  frequency  re-use  in  both  indoor  and  outdoor 
environments,  years  earlier  covered  in  Kinoshita  et  al.’s  “Frequency  common  use 
between  indoor  and  cellular  radio  research  on  frequency  channel  doubly  reused  cellular 
system”  (1989).  From  1996  to  present  there  has  been  significant  research  on  femtocells. 
Joseph  Boccuzzi  et  al.’s  “Femtocell  Design  and  Applications”  (2011)  and  Jie  Zhang  et 
al.’s  “Femtocells  Technologies  and  Deployment”  (2010)  are  some  of  the  most  recent  and 
extensively  cover  the  subject  of  femtocells  with  an  emphasis  on  the  deployment  and  use 
of  the  equipment  in  a  more  commercial  manner. 
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The  3ld  Generation  Partnership  Project  (3GPP),  a  partnership  that  produces 
technical  specifications  and  reports  pertinent  to  3ld  generation  cellular  systems,  has  issued 
numerous  releases  that  prepare  and  support  the  continuous  evolution  of  femtocells  (3GPP 
releases  9  and  10,  TS  25.467  and  25.306,  TR  21.905).  In  2007,  the  Femto  Forum  was 
created  to  promote  wide-scale  adoption  of  femtocells.  This  forum  has  played  a  role  in 
ensuring  that  the  standards  were  agreed  upon  and  released  to  the  public.  Publications 
from  this  forum,  “Interference  Management  in  UTMS  Femtocells”  (2010)  and 
“Regulatory  Aspects  of  Femtocells”  (2011),  speak  to  the  challenges  of  interference  and 
regulatory  issues  with  which  the  femtocell  community  is  currently  dealing.  There  are 
also  technical  literature  and  periodic  reviews  that  deal  heavily  with  LTE  prospects  and 
transmission  issues  (V.  Chandrasekhar  et  al.’s  “Femtocell  Networks:  a  Survey”  (2008), 
and  D.  Knisely  et  al.’s  “Standardization  of  Femtocells  in  3 GPP”  and  “Standardization  of 
Femtocells  in  3GPP2”  (both  in  2009).  In  2010,  methods  to  improve  joint  macro  level  and 
femtocell  level  frequency  assignments  and  alternate  optimized  frequency  reuse  schemes 
are  addressed  in  Y.  Haddad  et  al.’s  “Femtocell  SINR  Performance  Evaluation”  (2010). 
Additional  challenges  and  issues  relating  to  combining  and  synchronizing  signals  from 
other  base  stations  are  studied  in  S.  Kim  et  al.’s  “Performance  Analysis  of  LTE 
Enterprise  Femtocell  Using  Cooperative  Downlink  Transmission  Scheme”  (2011).  This 
study  stresses  the  need  to  utilize  the  LTE  FDM  scheme  to  get  overlapping  resources, 
which  lead  to  better  SINR  and  reinforced  signals. 

Analyses  of  mixed  Macro-cell  and  Femtocell  cases  are  seen  in  B.  Kaufman  et 
al.’s  “Femtocells  in  Cellular  Radio  Networks  with  Successive  Interference 
Cancellation”(201 1).  These  analyses  look  to  introduce  a  Femtocell  power  control 
process  that  does  not  require  coordination  with  macro-cells.  The  ultimate  goal  is  defining 
an  optimal  Macro-cell-to-Femtocell  hand-off.  Other  interference  issues  involved  with 
Heterogeneous  Networks  (HeTNeT)  are  referenced  in  D.  Lopez-Perez  et  al.’s  “Enhanced 
Intercell  Interference  Coordination  Challenges  in  Heterogeneous  Networks”  (2011). 
Lopez-Perez  et  al  deals  specifically  with  the  control  channel  degradation  problems  and 
the  application  of  different  power  control  techniques  in  Femtocells. 
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Recently,  numerous  publications  have  been  released  dealing  with  the  very 
important  issues  of  resource  assignments  and  optimization.  A  few  examples  of  these  are: 
G.  de  La  Roche  et  al.’s  “Selforganization  for  LTE  enterprise  femtocells”  (2010),  Y. 
Haddad  et  al.’s  “Analysis  of  an  Efficient  Channel  Assignment  Scheme  for  Femtocell” 
(2011),  F.  Tariq  et  al.’s  “Dynamic  Fractional  Frequency  Reuse  Based  Hybrid  Resource 
Management  for  Femtocell  Networks”  (2011),  S.  Das  et  al.’s  “  Issues  in  Femtocell 
Deployment  in  Broadband  OFDMA  Networks:  3GPP-LTE  a  case  study”  (2011),  and  X. 
Chu  et  al.’s  ’’Resource  Allocation  in  Hybrid  Macro/Femto  Networks”  (2010). 

D.  de  la  Roche  et  al.’s  “Selforganization  for  LTE  enterprise  femtocells”  (2010) 
discusses  methods  and  relating  problems  with  the  use  of  multiple  LTE  femtocells  in 
different  environments.  De  la  Roche  et  al  proposes  that  the  best  global  throughput  can  be 
achieved  by  a  specifically  proposed  self-organizing  network  technique.  The  need  for 
Femtocell  Access  Points  to  share  spectrum  and  several  algorithms  to  share  this  spectrum 
across  multiple  Femtocell  Access  Points  (FAPs)  are  explored  in  Y.  Haddad  et  al.’s 
“Analysis  of  an  Efficient  Channel  Assignment  Scheme  for  Femtocell”  (2011).  Haddad  et 
al  states  that  a  central  database  that  holds  infonnation  on  all  FAPs  in  most  cases  offers 
the  better  perfonnance.  Methods  of  using  Hybrid  Resource  Management  Algorithms 
(HRMA)  for  down-link  OFDMA  purposes  in  order  to  offer  better  performance  with  a 
larger  number  of  nodes  is  discussed  in  F.  Tariq  et  al.’s  “Dynamic  Fractional  Frequency 
Reuse  Based  Hybrid  Resource  Management  for  Femtocell  Networks”  (2011).  When 
referring  to  broadband,  S.  Das  et  al.’s  “Issues  in  Femtocell  Deployment  in  Broadband 
OFDMA  Networks:  3GPP-LTE  a  case  study”  (2011)  shows  the  benefits  of  the  co¬ 
existence  of  macro/micro-cells  and  co-channel  femtocells  in  OFDMA -FDD  systems. 

Femtocell  systems  are  addressed  in  a  broader  manner  in  numerous  research 
papers.  Some  of  the  relevant  publications  are:  S.  Hassan  et  al.’s  “Femtocell  versus  Wi-Fi 
-A  Survey  and  Comparison  of  Architecture  and  Performances”  (2009),  F.  Meshkati  et 
al.’s  “Mobility  and  Capacity  Offload  for  3G  UMTS  Femtocells”  (2009),  and  M.  Khan  et 
al.’s  “Local  IP  Access  (LIP A)  Enabled  3G  and  4G  Femtocell  Architectures”  (2011).  The 
latter  describes  several  architectures  for  different  LIPA  scenarios.  Trade  off  implications 
between  capacity  offload  and  UE  battery  life  with  regards  to  Femtocells  in  3GPP  UMTS 
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are  the  subject  of  F.  Meshkati  et  al.’s,  “  Mobility  and  Capacity  Offload  for  3G  UMTS 
Femtocells”  (2009). 

The  similarities  and  differences  between  Femtocells  and  technologies  such  as  Wi¬ 
Fi  is  the  subject  of  S.  Hassan  et  al.’s  “Femtocell  versus  Wi-Fi- A  Survey  and  Comparison 
of  Architecture  and  Performances”  (2009).  This  work  states  that  both  Femtocells  and 
Wi-Fi  can  provide  services  based  on  the  use  of  IP  networks  and  that  “evaluating  their 
performances  under  the  varied  conditions  of  IP  networks  is  an  interesting  area  of  future 
work.” 

E.  THESIS  ORGANIZATION 

Chapter  I  validated  the  need  for  this  research  by  providing  an  overview  of  the 
purpose  and  relevance  of  this  research.  With  its  brief  introduction  and  background  to  the 
subject  of  femtocells,  Chapter  I  is  intended  to  point  out  that  there  has  been  little  research 
with  regards  to  comparing  femtocell  and  Wi-Fi  capabilities  and  performance.  The 
chapter  also  contains  a  literature  review  of  relative  femtocell  research  and  concludes  with 
a  discussion  of  how  this  body  of  work  is  organized. 

Chapter  II  presents  a  brief  history  of  femtocell  technology.  Chapter  II  goes  on  to 
list  several  issues  and  challenges  associated  with  femtocell  deployment.  These  included 
quality  of  service,  freq uc n c y /b a n d w i t h ,  interference,  handover,  regulatory,  and  security 
challenges  and  issues.  The  chapter  goes  on  to  describe  basic  femtocell  and  Wi-Fi 
architectures. 

Chapter  III  presents  a  description  of  the  methodology  and  experiments  that  will  be 
conducted  within  this  research.  Chapter  III  began  with  establishing  baseline  testing  in 
both  ideal  and  non-ideal  conditions.  The  chapter  then  goes  on  to  describe  testing  in  the 
areas  of  Internet  accessing  via  HTTP,  and  downloading  of  various  sized  fdes,  various 
sized  RTP  file  streaming,  and  VoIP. 

Chapter  IV  lists,  describes,  and  summarizes  the  data  collected  in  Chapter  III. 
Chapter  IV  goes  on  to  point  out  the  key  findings  and  the  results  of  the  tests  performed  in 
Chapter  III. 
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Chapter  V  provides  an  overall  conclusion  to  the  research  study.  The 
chapter  revisits  the  intent  of  the  research  to  ensure  that  all  objectives  set  forth  were 
adequately  addressed.  The  chapter  concludes  by  highlighting  recommendations  and 
potential  future  research  topics  relating  to  this  research. 
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II.  FEMTOCELL  HISTORY,  ISSUES,  AND  CHALLENGES 


A.  A  BRIEF  HISTORY  OF  FEMTOCELLS 

The  actual  term  femtocell  is  used  to  describe  a  coverage  area,  scale,  or  size.  As 
shown  in  Figure  1,  the  macrocell  is  the  largest  level  cell  and  it  provides  the  widest  range. 
The  macrocell  is  found  in  most  rural  areas  and  can  be  located  along  major  highways. 
The  next  smaller  size  cell  is  the  microcell.  It  is  used  in  very  densely  populated  areas 
(mostly  urban)  like  cities  and  large  towns.  Within  these  cells  is  the  picocell,  which  is  for 
areas  that  are  even  smaller.  Picocells  are  often  found  in  large  office  buildings,  industrial 
areas,  and  commercial  areas  (i.e.,  shopping  centers  and  malls).  The  smallest  cell  is  the 
femtocell.  Femtocells  can  be  found  in  a  person’s  home  or  an  individual  office. 

Research  into  “small  cells”  can  be  found  in  literature  as  early  as  1984.  For 
instance,  in  his  article  “Small-Cell  Mobile  Phone  Systems,”  Arthur  Stockton  describes 
systems  that  have  “direct  access  to  the  land  telephone  network  and  are  designed  to 
connect  any  mobile  phone  to  any  other  phone,  mobile  or  not.”  In  the  1990’s  there  was 
increasing  demand  for  cellular  services  and  as  a  result  the  macrocells  were  being 
overloaded. 


Figure  1.  Macro-cell,  micro-cell,  pico-cell,  and  femto-cell  ranges 
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This  led  to  the  development  by  Southwest  Bell  and  Panasonic  of  a  method  of 
reusing  the  same  frequencies  as  outdoor  (macrocellular)  cellular  systems  to  provide 
wireless  communications  inside  a  building.  This  was  accomplished  by  using  a  wired 
backhaul.  Even  though  the  technology  wasn’t  quite  there  to  support  the  IP  backhaul  and 
it  was  very  costly,  it  was  the  first  actual  femtocell  type  network. 

Over  the  last  decade  there  has  been  a  dramatic  increase  in  consumer  demand  for 
increased  capabilities  through  mobile  means.  According  to  Cisco  Visual  Networking 
Index  (Cisco  white  paper:  Forecast  and  methodology,  2011-2016),  the  amount  of  global 
mobile  data  traffic  in  201 1  has  more  than  doubled  for  the  fourth  year  in  a  row,  and  global 
mobile  data  traffic  in  2011  was  over  eight  times  greater  than  the  total  global  internet 
traffic  in  2000.  With  this  incredible  growth,  the  need  for  new  cellular  architecture  with 
greater  capacity  was  necessary.  Fortunately,  the  development  of  4G  standards  that  are 
based  on  Orthogonal  Frequency-Division  Multiple  Access  (OFDMA)  and  IP  have 
provided  a  more  efficient,  low  cost,  platfonn  for  femtocells. 

Current  technology  has  introduced  automatic  configuration  and  self-optimization 
capabilities  in  femtocells  making  them  user  friendly  and,  ultimately,  sold  in  a  plug-and- 
play  type  product.  They  also  have  the  ability  to  automatically  integrate  into  macro- 
cellular  networks.  As  a  result,  over  the  last  four  years  major  femtocell  deployments  by 
the  biggest  cellular  companies  in  the  world  has  occurred.  Sprint,  Verizon,  AT&T,  and 
others  worldwide  now  offer  femtocells  compatible  with  their  underlying  radio¬ 
infrastructures. 

B.  FEMTOCELL  ISSUES  AND  CHALLENGES 

1.  Quality  of  Service  Issues 

The  tenn  Quality  of  Service  (QoS)  refers  to  the  requirements  that  are  imposed  by 
IEEE  802.11  on  all  aspects  of  an  Internet  connection.  Some  of  these  requirements  are 
adequate  signal-to-noise  ratio  (SNR),  frequency  responses,  loudness  levels,  response 
time,  loss,  etc.  The  intent  is  to  guarantee  a  standardized  level  of  quality  and  performance 
for  the  consumer’s  data  flow  needs.  The  issue  with  QoS  for  femtocells  is  that  in  order  to 
achieve  QoS  requirements  there  often  needs  to  be  hardware  changes.  A  possible  solution 
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may  be  to  use  a  traffic  classifying  service  like  Differentiated  Services  (DiffScrv)  or 
Integrated  Services  (IntServ).  IntServ  improves  QoS  by  having  applications  use  resource 
reservation  protocol  (RSVP)  to  improve  requests  and  reserve  resources  through  a 
network.  DiffScrv  prioritizes  packets  according  to  the  type  of  service  they  desire. 
Routers  and  switches  can  prioritize  these  to  improve  quality.  WiFi,  however,  already 
must  comply  with  IEEE  802.1  lQoS  standards,  and  currently  has  established  mechanisms 
in  place  to  ensure  QoS. 

2.  Frequency  /  Bandwidth  Issues 

The  electromagnetic  spectrum  is  a  scarce  and  crowded  resource.  Femtocells 
operate  on  the  same  licensed  spectrum  that  is  allocated  to  cellular  service  providers.  To 
deal  with  this  overcrowding  issue  two  methods  have  been  used:  the  Co-channel 
Frequency  Deployment  and  Orthogonal  Channel  Deployment.  The  Co-channel 
Frequency  Deployment  simply  allows  the  femtocell  and  the  cellular  macro-cell  to  use  the 
same  frequency  band.  With  co-channel  use,  however,  there  are  identified  interference 
issues.  Orthogonal  Channel  Deployment  is  in  many  ways  the  opposite  of  Co-channel 
Frequency  Deployment.  In  this  method  macro-cells  and  femtocells  use  separate 
channels.  The  advantage  to  this  method  is  that  there  is  less  potential  for  interference,  the 
disadvantage  is  a  reduction  in  the  overall  system  capacity. 

WiFi  networks  use  different  Industrial,  Scientific,  and  Medical  (ISM)  frequency 
bands.  These  unlicensed  ISM  bands  are  operated  independently  of  any  specific  cellular 
service  and  are  available  for  public  use.  This  however  may  also  lead  to  interference 
problems  when  too  many  WiFi  devices  are  located  near  each  other  using  the  same  band. 

3.  Interference  Issues 

As  stated  earlier,  there  is  limited  spectrum  on  which  cellular  systems  can  operate 

and  the  spectrum  is  controlled  by  licensing.  Femtocells  utilize  the  spectrum  already 

licensed  for  cellular  providers.  Thus,  interference  is  a  key  issue  associated  with 

femtocells.  When  multiple  femtocell  devices  are  being  serviced  by  the  same  macro-cell 

there  can  be  adjacent  channel  interference.  There  can  also  be  interference  issues  when 

several  femtocell  devices  are  used  in  close  proximity  to  each  other,  regardless  of  whether 
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or  not  they  are  serviced  by  the  same  macro-cell.  Generally  though,  femtocells  are  used  in 
areas  of  poor  or  limited  cellular  coverage  and  in  these  cases  interference  from 
overcrowded  networks  is  not  an  issue.  Also,  a  benefit  of  the  low  power  output  of  the 
femtocell  is  that  multiple  femtocell  devices  would  have  to  be  very  close  to  each  other  to 
cause  interference.  WiFi  devices  can  also  face  similar  interference  issues  stemming  from 
the  fact  that  all  the  WiFi  devices  are  working  on  the  same  unlicensed  band,  are  often  in 
the  same  vicinity  as  other  WiFi  access  points  or  user  devices,  and  use  a  very  limited 
number  of  non-overlapping  channels  -  specifically,  three  of  the  eleven  available  in  the 
U.S.  (of  the  twelve  overseas).  This  latter  fact  impacts  the  utility  of  WiFi  in  congested 
areas. 


4.  Handover  Challenges 

When  a  mobile  device  in  a  WiFi  network  moves  to  the  outer  edge  of  its  Received 
Signal  Strength  (RSS)  limit  it  needs  to  perform  a  “handover”  of  connection  from  one 
access  point  to  another.  The  major  concern  for  femtocell  handover  is  that  the  coverage 
area  of  an  individual  femtocell  is  very  small.  For  this  reason,  it  becomes  essential  that 
there  is  a  seamless  handover  to  and  from  femtocells  so  the  user  can  maintain  continuous 
signal  connectivity.  There  are  generally  three  types  of  handovers  for  both  WiFi  and 
Femtocells.  The  first  is  a  simple  base  station  to  base  station  handover  where  a  user 
moves  from  the  range  of  one  base  station  to  another.  The  second  occurs  between  base 
stations  and  Femto  Access  Points  (FAPs). 

The  base  station  to  FAP  handover  happens  when  the  mobile  user  moves  from  an 
outdoor  area  to  an  indoor  area.  When  the  user  starts  outdoors  it  sends  a  request  to  a 
cellular  base  station  and  when  the  user  then  moves  indoors  the  FAP  will  accept  the 
request  and  pick  up  the  signal.  For  this  to  work  there  has  to  be  synchronization  between 
the  FAP  and  the  cellular  base  station. 

The  final  handover  scenario  is  where  the  user  moves  from  one  FAP  to  another. 
This  generally  happens  when  there  are  multiple  FAPs  in  the  same  vicinity,  in  an  office 
building  for  example.  The  challenge  associated  with  handovers  for  femtocells  is  that  they 
are  not  usually  connected  to  a  network  enviromnent  where  mobility  is  addressed,  (again 
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as  in  an  office  building  where  mobility  outside  the  building  isn’t  a  concern).  Due  to  the 
fact  that  the  femtocell  must  be  associated  with  an  IP  address,  whe  a  user  is  mobile  the  IP 
addresses  would  have  to  change. 

5.  Regulatory  Challenges 

One  of  the  biggest  differences  between  WiFi  and  femtocells  is  the  fact  that  WiFi 
operates  in  an  unlicensed  spectrum  while  femtocells  operate  in  a  licensed  spectrum  and 
require  regulatory  approval.  This  becomes  an  issue  because  the  spectrum  and  radio 
regulations  will  vary  from  one  country  to  the  next.  International  agreements  can  also  be 
involved  when  a  user  takes  their  femtocell  from  one  nation  to  another.  In  a  licensed 
spectrum  the  provider  pays  substantial  sums  to  be  able  to  use  a  portion  of  the  spectrum 
exclusively  and  regulators  will  enforce  transgressions.  This  means  that  a  femtocell 
operator  could  not  just  move  their  femtocell  to  another  country  and  operate  it.  The 
varying  spectrum  allocations  from  one  country  to  another  can  also  prevent  unauthorized 
usage. 

A  femtocell  has  several  means  to  identify  where  it  is.  The  first  is  a  Global 
Positioning  System  (GPS)  receiver  that  is  built  into  the  femtocell.  This  immediately 
identifies  the  location  of  the  femtocell.  Another  means  is  by  mapping  its  IP  address  to 
the  femtocell’s  originating  country.  A  femtocell  also  can  sense  other  cell  site  identities  in 
its  area  and  can  identify  its  “neighborhood.”  If  a  femtocell  sees  that  it  is  in  an 
unauthorized  area  it  can  disable  itself  or  notify  the  provider. 

Due  to  the  regulatory  issues  operators  cannot  use  their  femtocells  in  frequency 
spectrum  that  they  do  not  own  and  control.  Some  large  providers  (Verizon,  AT&T,  T- 
Mobile)  may  have  operations  in  several  different  countries  and  therefore  they  license  the 
spectrum  in  those  countries.  A  femtocell  user  with  one  of  these  companies  would  still 
not  be  able  to  use  their  femtocell  in  these  countries  because  the  spectrum  allocation  may 
be  different  and  the  femtocell  would  still  broadcast  its  original  identity  (trying  to  connect 
to  its  home  network).  Many  femtocells  have  2G  and  3G  receivers  that  can  scan  for 
signals  from  external  cellsites  and  can  detennine  the  country  in  which  it  is  located  and 
what  networks  are  available.  The  surrounding  cellsite  identities  will  change  if  the 
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femtocell  is  moved.  This  could  interfere  with  local  mobile  phone  users  who  could 
possibly  pick  up  the  signal,  and  would  also  cause  unnecessary  hand-offs  that  would 
reduce  signal  strength. 

6.  Security  Challenges 

The  security  of  a  device  or  network  is  always  a  paramount  concern  for  users, 
especially  on  a  wireless  medium.  There  are  three  major  security  vulnerability  concerns 
for  femtocell  network  technology.  The  first  comes  from  the  wireless  link  into  the 
femtocell.  According  to  a  technical  white  paper  from  Picochip  (2011),  it  is  possible  for 
external  wireless  transmissions  to  potentially  gain  unauthorized  access  to  the  femtocell. 
The  second  concern  is  the  backhaul  link  that  is  used  between  the  femtocell  and  the 
gateway  into  the  service  provider’s  core  network  (the  Internet  link).  The  third  concern  is 
the  femtocell  itself,  as  it  is  potentially  possible  for  nefarious  network  users  to  get  into  the 
femtocell  and  take  control  of  it  remotely. 

There  are  several  ways  to  prevent  or  counter  these  security  issues.  The  first  is  to 
ensure  secure  authentication.  Authentication  needs  to  be  required  by  the  service  provider 
or  the  operator  to  correctly  identify  valid  femtocells  within  the  network.  Another  means 
of  ensuring  security  is  the  use  of  Internet  Protocol  Security  (IPsec).  IPsec  is  a  protocol 
for  securing  IP  communications  by  authenticating  and  encrypting  each  IP  packet.  It  also 
establishes  mutual  authentication  and  provides  cryptographic  keys  for  additional  security. 
Extensible  Authentication  Protocol  (EAP)  is  an  authentication  framework  for  wireless 
networks  and  also  provides  a  means  of  ensuring  wireless  security. 

C.  BASIC  FEMTOCELL  ARCHITECTURE 

A  basic  femtocell  network  architecture,  as  shown  in  Figure  2,  is  generally 
comprised  of  three  elements:  a  Femtocell  Access  Point  (FAP),  a  security  gateway,  and  a 
femtocell  management  system.  The  FAP  base  station  also  requires  a  means  of 
connecting  to  the  Internet,  typically  through  a  broadband  Internet  connection  (DSF,  cable 
modem,  or  direct  ISP  access). 

The  Femtocell  Access  Point  is  basically  a  small  scale  cellular  base  station. 

It  is  the  primary  node  in  the  network  that  connects  the  user  to  the  network,  and  can  be 
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used  in  stand-alone  or  integrated  configurations.  When  used  in  the  stand-alone 
configuration  the  FAP  is  connected  directly  to  the  user’s  router,  whereas  in  an  integrated 
method  the  FAP  has  its  own  built-in  router.  A  typical  femtocell  access  point  base  station 
will  transmit  100  mW  of  power,  has  a  data  rate  of  between  7.2  and  14.4  Mbps,  operates 
at  between  1.9  and  2.6  GHz,  and  has  an  effective  range  for  high  performance  of 
approximately  100  feet  (dependent  upon  location  and  interference  issues).  Note  that  this 
data  rate  is  comparable  to  IEEE  802.3  10BASET  connections  (traditional  “Ethernet”). 


Figure  2.  Basic  femtocell  architecture 


In  all  systems  pertinent  to  operations  and  support,  security  is  a  very  important 
issue.  In  a  femtocell  architecture  the  security  gateway  is  a  network  node  that  provides  a 
secure  means  to  access  the  Internet.  The  security  gateway  uses  Internet  Protocol  Security 
(IPSec)  and  Internet  Key  Exchange  (IKEv2)  Internet  security  protocols  for  encryption 
support  and  for  the  authentication  and  authorization  of  the  femtocells.  Femtocell 
security  gateways  are  network  nodes  that  have  been  designed  for  use  in  carrier-type 
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networks  and  meet  availability,  scalability,  and  network  management  security 
requirements. 

The  femtocell  management  system  is  arguably  the  most  important  element  in  the 
femtocell  architecture.  A  femtocell  management  system  must  comply  with  Technical 
Report  069  (TR-069),  which  is  a  protocol  for  communication  between  Customer  Premise 
Equipment  (CPE)  and  Auto-Configuration  Servers  (ACS)  that  encompasses  secure  auto¬ 
configuration  as  well  as  other  CPE  management  functions  within  a  common  framework. 
The  femtocell  management  system  plays  a  critical  role  in  the  operational  management, 
provisioning,  and  activation  of  the  femtocells.  It  is  the  femtocell  management  system 
that  allows  the  operator  to  control  the  device  remotely  ensuring  that  it  is  in  compliance 
with  local  regulations. 

For  Internet  connectivity,  femtocells  connect  to  the  mobile  operator’s  network  via 
a  standard  broadband  connection,  such  as  DSL,  fiber,  or  cable.  The  data  to  and  from  the 
femtocell  is  carried  over  an  IP  technology-based  network  provided  by  an  Internet  Service 
Provider.  For  wireless  (mobile  device)  users,  the  connection  to  the  femtocell  is  done  via 
the  normal  cellular  service  technologies  just  as  if  they  were  using  a  conventional  macro- 
cellular  network  to  connect. 

D.  BASIC  WI-FI  ARCHITECTURE 

A  wireless  local  area  network  (WLAN)  is  a  collection  of  wireless  devices  that 
will  maintain  connectivity  with  each  other  while  transferring  data.  The  WLAN  works  in 
three  basic  configurations:  peer-to-peer,  bridge,  and  wireless  distribution  system.  Peer- 
to-peer  configuration  is  where  each  computer  in  the  network  can  act  as  a  client  or  server 
for  the  other  computers  in  the  network.  This  allows  them  shared  access  to  files  (such  as 
audio,  video,  data,  etc.)  and  peripherals  without  needing  a  central  server.  A  bridge 
configuration  is  used  to  connect  networks.  This  is  done  by  use  of  a  wireless  Ethernet 
bridge,  providing  the  connection  for  devices  to  a  wireless  network.  The  wireless 
distribution  system  enables  the  wireless  inter-connection  of  the  access  points  within  a 
network.  This  allows  a  wireless  network  to  be  expanded  through  the  use  of  multiple 
access  points  linked  together.  Generally,  a  WLAN’s  signal  can  reach  to  500  feet  indoors 
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and  approximately  1000  feet  outdoors.  WiFi  operates  in  the  unlicensed  2.4  GHz  or  5.8 
GHz  ISM  Band.  WiFi  transmissions  are  essentially  FM  transmission,  in  that  the 
frequency  is  changed  to  transmit  data.  The  2.4  GHz  spectrum  is  shown  in  Figure  3. 


Figure  3.  2.4  GHz  Wi-Fi  channels  from  Gauther,  M.,  Wireless  networking  in  the 

developing  world  (2009) 

WiFi  is  a  limited  range  wireless  networking  protocol  based  on  the  IEEE  802.11 
standards.  Having  WiFi  connectivity  allows  a  user  to  transfer  data  at  the  speed  of 
broadband  using  radio  waves  rather  than  a  wired  or  cabled  infrastructure.  A  short-range 
wireless  network  (often  referred  to  as  a  WiFi  network  or  Wireless  Local  Area  Network 
(WLAN))  is  set  up  by  using  radio  signal  frequency  to  communicate  among  computers 
and  other  wireless-enabled  devices.  The  main  architectural  components  of  a  wireless 
network  are  the  wireless  router  (access  point),  WiFi  cards,  safeguards,  and  one  or  more 
wireless  clients.  In  simplest  terms,  an  Access  Point  (AP)  is  a  wireless  LAN  transceiver, 
or  “base  station,”  that  can  connect  one  or  many  wireless  devices  simultaneously  to  the 
Internet.  WiFi  cards  are  installed  in  client  devices  and  accept  the  wireless  signal  and 
relay  information.  Safeguards  are  firewalls  or  anti-virus  software  products  that  protect 
networks  and  help  to  keep  information  secure. 

As  shown  in  Figure  4,  the  basic  WiFi  architecture  starts  with  a  station.  This  is 

essentially  a  computer  that  can  be  either  mobile  or  fixed.  A  Basic  Service  Set  (BSS)  is 

created  when  two  or  more  stations  come  together  in  order  to  communicate  with  each 

other.  According  to  IEEE  802.11,  there  are  two  types  of  operating  modes:  infrastructure 
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mode,  and  ad  hoc  mode.  Infrastructure  mode  is  used  to  connect  a  computer  with  a 
wireless  network  adapter  (or  wireless  client)  to  a  wired  network.  This  is  accomplished 
through  a  wireless  router  or  access  point.  Ad  hoc  mode  is  used  to  connect  wireless 
clients  directly  together.  This  method  does  not  use  a  wireless  router  or  access  point.  The 
ad-hoc  network  refers  to  when  a  BSS  is  not  connected  to  an  Internet  interface  device  and 
it  is  then  referred  to  as  an  independent  BSS  (IBSS). 

When  two  or  more  basic  service  sets  need  to  be  connected  it  is  done  through  a 
Distribution  System  (DS).  The  DS  increases  network  coverage  by  allowing  the  wireless 
network  to  be  expanded  using  multiple  access  points  without  needing  a  wired  backbone 
to  link  them. 


Figure  4.  Basic  Wi-Fi  network 


In  this  chapter  we  discussed  the  functionality  of  femtocells,  and  introduced  the 
reader  to  some  of  the  prevalent  issues  and  challenges  of  femtocell  technology.  These 
items  need  to  be  resolved  for  the  femtocell  to  be  an  accepted  and  widely  used  technology. 
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The  key  is  to  ensure  that  femtocells  are  scalable,  and  easily  integrated.  Femtocells  also 
must  be  robust  enough  to  successfully  deal  with  security,  regulatory,  and  interference 
issues.  This  chapter  ended  with  a  basic  description  of  both  femtocell  and  WiFi 
architectures.  With  a  basic  understanding  of  the  history,  issues,  and  architecture  of 
femtocells  we  now  turn  to  the  purpose  of  this  work,  which  is  to  compare  and  evaluate  the 
performances  between  femtocells  and  WiFi  networks. 
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III.  METHODOLOGY 


A.  INTRODUCTION 

The  approach  to  this  work  is  directly  linked  to  the  purpose,  objectives,  and  scope 
listed  in  Chapter  I.  There  is  currently  very  little  research  published  specifically 
developing  an  analysis  of  Femtocell  versus  WiFi  in  terms  of  their  respective  performance 
and  capabilities.  In  this  chapter,  we  provide  the  methodology  for  the  tests  and  extensive 
experimentation  conducted  with  respect  to  these  technologies. 

Our  research  includes  numerous  testing  tools  that  provide  multiple  measurements. 
These  are  evaluated  and  compared  as  to  suitability  for  the  experiments  conducted  herein. 
To  compare  the  performances  between  Femtocell  and  WiFi,  we  must  start  first  with  a 
stable  environment  where  the  testing  can  be  repeated  with  no  outside  interference  nor 
variation.  In  our  study,  we  use  a  single  laptop  computer  that  functions  as  a  client.  The 
laptop  is  alternately  connected  to  the  Internet  through  either  a  router  and  a  Femtocell  or  a 
wireless  router  alone.  When  using  the  Femtocell,  the  laptop  is  connected  to  the  Internet 
via  a  mobile  hotspot  provided  by  a  cellular  “smart”  phone. 

Femtocells  are  base  stations  that,  by  design,  connect  automatically  to  a  cellular 
mobile  operator  network.  This  internal  connection  process  renders  it  impossible  to 
manually  create  and  manipulate  your  own  network.  Thus,  due  to  its  server-client 
architecture,  communication  among  devices  directly  through  the  femtocell  is  not  possible 
and  can  only  be  accomplished  through  the  transfer  of  data  from  the  client,  to  the  network, 
and  through  the  server.  Due  to  these  facts,  we  chose  to  use  a  single  laptop  computer  as  a 
client  with  both  access  to  a  Wi-Fi  router  and  a  mobile  hotspot.  As  this  is  how  a  mobile 
phone  accesses  the  Internet  this  is  actually  beneficial  for  our  research,  as  it  provides  a 
more  realistic  environment  for  testing. 

B.  TECHNICAL  SPECIFICATIONS 

The  devices  used  in  this  research  can  be  described  as  micro-environmental 
devices.  This  means  that  they  are  all  located  indoors  and  are  purchased,  owned  and 
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operated  by  the  average  Internet  user.  They  include  a  wireless  router  (802.11  wireless 
access  point),  a  Femtocell  (Verizon  Network  Extender),  a  Motorola  Droid  RAZR  4G 
cellular  phone  (with  its  mobile  hotspot  enabled),  and  network  endpoints  (client  and  server 
nodes).  A  comparison  of  the  general  specifications  for  Wi-Fi  and  Femtocells  can  be  seen 
in  Table  1. 


Wifi 

Femtocell 

Data  Range  Capabilities 

1 1  and  54  Mbps 

7.2  -  14.4  Mbps 

Operating  Frequency 

2.4  and  5  GHz 

1.9 -2.6  GHz 

Power  Output 

100,  200  mW 

10,  100  mW 

Range 

100 -200  m 

20  -30  m 

Services  Provided 

Voice  &  Data 

Voice  &  Data 

Table  1.  General  Wi-Fi  femtocell  specifications 


1,  Wireless  Router 

A  wireless  router  is  a  device  that  provides  wireless  signals  for  connecting  network 
devices  that  have  wireless  adapters  (Table  2).  The  purpose  of  the  wireless  router  is  to 
send  wireless  signals  that  can  be  interpreted  by  the  wireless-enabled  network  clients  for 
communicating  data  and  information.  Routers  collect  signals  and  convert  them  into 
wired  signals  and  send  them  over  the  Local  Area  Network  (LAN).  A  wireless  router  will 
generally  have  a  20  -  30  meter  indoor  range  and  approximately  80  -  100  meter  outdoor 
range.  Routers  can  usually  connect  several  wireless  devices  within  their  area  of 
coverage. 
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Model 

Neteear  WGR614  (v7) 

Frequencies: 

2.4 -2.5  GHz 

Network  Speed/Data  Rates: 

54  Mbps 

Antenna: 

2  dBi 

Encryption: 

64  bit,  128  bit,  and  152  bit,  WEP  encryption, 
WPA  (WiFi  Protected  Access) 

Data  and  Routing  Protocols: 

TCP/IP,  RIP-1,  RIP-2,  DHCP,  PPP  over 
Ethernet  (PPPoE) 

Interface  Specifications: 

LAN:  10BASE-T  or  100BASE-TX,  RJ-45 

WAN:  10BASE-T  or  100BASE-TX,  RJ-45 

Data  Encoding: 

802.11b:  Direct  Sequence  Spread  Spectrum 
(DSSS) 

802. 1  lg:  Orthogonal  Frequency  Division 
Multiplexing  (OFDM) 

Table  2.  Wireless  router  specifications  from  Netgear  Support  www.Netgear.com 


2.  Femtocell 

The  femtocell  is  a  small  cellular  base  station  that  is  designed  for  use  in  small 
areas,  like  homes  and  buildings  (Table  2).  It  works  by  connecting  to  a  service  provider’s 
network  through  broadband  networks  (DSL  or  cable).  Femtocells  have  a  range  of  20  - 
30  meters,  and  can  generally  support  up  to  six  devices.  A  comparison  of  all 
commercially  available  Femtocells  is  provided  in  Figure  5.  For  our  testing  we  will  be 
using  the  Samsung  Wireless  Network  Extender  provided  by  Verizon  (Table  3)  connected 
directly  to  the  wireless  router  via  an  Ethernet  cable. 
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Carrier 

AT&T 

Verizon 

Sprint 

T-Mobile 

Solution 

Femtocell  -  "3G 
MicroCell" 

Femtocell 

"Network 

Extender" 

Femtocell 

"Airave" 

UMA 

"FlotSpot@Home" 

Branding 

Cisco 

Samsung 

Samsung 

NA 

Technology 

3G  UMTS/HSPA 
for  voice  and  data 

2.5G  CDMA 

2000  lxRTT 

2.5G  CDMA 
2000  lxRTT 

UMA  voice  over 
WiFi 

Simultaneous 

Calls 

4  Simultaneous 

3  Simultaneous 

3  Simultaneous 

NA 

Standby 

Approved 

Callers 

10 

100 

50 

NA 

Data  Bitrate 

3.6  megabits/s 

(HSDPA3.6) 

144  kilobits/s 

144  kilobits/s 

NA 

GPS  Fix 

Required 

Yes 

Yes 

Yes 

NA 

Hand-On/Hand- 

Off 

No/Yes 

No/Yes 

No/Yes 

Inter  AP 

Flandover/Yes 

Coverage 

5000  square  feet 

5000  square  feet 

5000  square  feet 

WiFi  AP  range 

Figure  5.  Femtocells;  A  comprehensive  exploration,  www.anandtech.com,  from 

Brian  Klug  (4/1/2010) 


Model 

Samsung  Wireless  Network  Extender  (SCS- 

2U01) 

Frequencies: 

800/1900  MHz 

Air  Interface: 

CDMA2000  lx  Rel  0 

CDMA2000  EvDO  0/A 

Traffic  Channel: 

Up  to  six  simultaneous  users  (a  seventh  is 
reserved  for  emergency  calls) 

Transmission: 

10/100  Base-T  Ethernet/Network 

Standards: 

IEEE  802.3,  IEEE  802.3u  for  Ethernet 

IEEE  802.1  lg,  IEEE  802.1  lb  for  Wireless 

Power  Range: 

10  mW  to  30  tnW 

Table  3.  Samsung  wireless  network  extender  femtocell  specifications 
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3.  Cellular  Smartphone 

In  this  research,  we  use  a  Motorola  Droid  RAZR  cellular  phone  with  the  Mobile 
Hotspot  application  enabled  with  the  specifications  found  in  Table  4.  The  Motorola 
Droid  RAZR  is  a  4G  capable  LTE  smartphone  that  has  a  built  in  Mobile  Hotspot 
application.  A  mobile  hotspot  is  a  means  of  allowing  the  mobile  phone  to  act  as  a  Wi-Fi 
access  point,  providing  a  network  access  to  nearby  computers,  Tablets  and  other  Wi-Fi 
capable  devices.  The  Motorola  Droids  mobile  hotspot  application  allows  the  phone  to 
connect  to  a  mobile  data  network  and  then  act  as  a  Wi-Fi  router,  distributing  the 
bandwidth  to  nearby  clients.  Any  Wi-Fi  enabled  computer  or  mobile  device  can  connect 
to  the  network  that  the  mobile  hotspot  provides.  A  significant  advantage  to  the  mobile 
hotspot  provided  by  the  Droid  RAZR  is  that  it  is  completely  mobile.  The  cellular  phone 
receives  its  signal  through  the  nearest  macro-cell  station.  Using  this  method,  however, 
requires  the  user  is  in  the  provider’s  area  of  coverage.  The  connection  speed  provided  by 
the  mobile  hotspot  depends  on  many  variables,  including  the  cell  network  to  which  you 
are  connecting,  how  far  you  are  from  the  network’s  closest  transmission  tower  and  how 
congested  the  shared  service  may  be  at  the  time.  A  typical  cellular  phone  mobile  hotspot 


has  a  range  of  approximately  100  feet. 


Model 

Motorola  Droid  RAZR  XT912 

General: 

2G  GSM  850/900/1800/1900  CDMA  800/1900 

3G  HSDPA  850/900/1900/2100  CDMA2000  lxEV- 
DO 

4G  LTE  700  MHz  Class  13-For  Verizon 

Memory: 

16  GB  storage,  1  GB  RAM 

Data: 

GPRS:  Class  12,  32-48  kbps 

EDGE:  Class  12 

Speed:  Rev.  A,  up  to  3.1  Mbps,  LTE,  HSDPA, 
HSUPA 

WLAN:  Wi-Fi  802.1 1  b/g/n,  DLNA,  Wi-Fi  hotspot 
Bluetooth:  v4.0  with  LE+EDR 

Operating  System: 

Andoid  OS,  v4.0.4  (Ice  Cream  Sandwich) 

Browser: 

HTML,  Adobe  Flash 

CPU: 

Dual-core  1 .2  GHz  Cortex- A9 

Chipset: 

TI  OMPA  4430 

Java: 

Via  Java  MIDP  emulator 

GPS: 

With  A-GPS  support 

Table  4.  Droid  RAZR  XT912  specifications 
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4. 


Network  Endpoint 


A  network  endpoint  is  a  device  that  enables  a  user  to  access  network  services.  In 
this  research  we  use  the  Motorola  Droid  RAZR  and  a  Hewlett-Packard  laptop  computer 
as  seen  in  Table  5.  The  Droid  RAZR  will  create  a  mobile  hotspot,  and  the  HP  laptop  will 
alternately  utilize  the  created  hotspot  network  and  the  router  created  Wi-Fi  networks  to 
run  the  tests  needed  to  measure  and  compare  the  performances  and  capabilities  of  the  two 
technologies. 


Model: 

Hewlett-Packard  HP  G62  Notebook  PC 

Microprocessor: 

2.50GHz  VISION  Technology  from  AMD  with  AMD 
Turion  II  Dual-Core  Mobile  Processor  N530 

Memory: 

4GB  DDR3  System  Memory  (2  DIMM) 

8MB  (max  memory) 

Video  Graphics: 

ATI  Mobility  Radeon  HD  4250  Graphics 

Video  Memory: 

Up  to  1917  MB 

Hard  Drive: 

320  GB  (5400RPM) 

Network  Card: 

Integrated  10/100  Ethernet  LAN 

Wireless  Connectivity: 

802.11  b/g/n  WLAN 

Operating  System: 

Windows  7  Home  Premium  64-bit 

Table  5.  Network  endpoint  specifications 


C.  TESTING 

The  ultimate  goal  of  this  research  is  to  evaluate  the  utility  of  Femtocells  by 
comparing  their  perfonnance  to  that  of  a  traditional  wireless  Wi-Fi  network.  To 
accomplish  this,  our  research  evaluated  the  performance  and  capabilities  of  these  two 
technologies  through  several  use-cases,  applications,  and  scenarios.  Our  research 
performed  baseline  tests  and  assessment  of  the  basic  perfonnances  of  these  two 
technologies  in  ideal  conditions.  These  tests  are  described  below.  We  then  transitioned  to 
a  realistic  environment  evaluation.  The  realistic  environment  tests  were  conducted  in  less 
than  ideal  settings  that  involve  both  line-of-sight  obstacles  and  distance  from  the  access 
point  or  femtocell.  Support  for  Internet  applications,  such  as  browsing  or  “surfing” 
through  HTTP  protocol,  RTP  protocol  applications  for  streaming,  and  transfer  of 
audio/video  files  was  also  conducted.  Finally,  we  evaluated  the  VoIP  supporting 

protocols  offered  by  both  Femtocell  and  Wi-Fi  provisioned  networks. 
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1.  Testing  terms 

Ping  is  basically  the  process  of  sending  an  echo-request  packet  from  the  user’s 
computer  to  a  different  or  remote  computer  (or  server).  The  time  between  the 
transmission  of  the  request  (or  ping)  and  the  receipt  of  the  associated  echo-reply  is  a 
measurement  of  the  latency  of  the  connection.  Ping  is  measured  in  milliseconds  (ms).  If 
the  user  experiences  a  delayed  response  in  Internet  applications  it  could  be  due  to  a 
higher  than  desired  network  latency.  Latency  is  a  term  that  basically  means  the  delay 
during  the  performance  of  a  given  operation.  Latency  is  used  to  describe  any  type  of 
delay  that  occurs  during  the  transmission  or  processing  of  data  packets,  such  as 
transmission,  propagation,  processing,  or  queuing. 

Jitter  is  the  variance  in  measuring  successive  ping  tests.  A  reading  of  zero  in  a 
jitter  test  means  that  the  results  were  exactly  the  same  every  time.  A  score  above  zero 
indicates  the  amount  by  which  they  varied.  The  lower  the  jitter  value  the  better  the 
connection  service  quality  for  applications  sensitive  to  delay. 

Packet  Loss  is  the  term  used  to  refer  to  unsuccessful  transmission  of  “packets”  of 
data.  Having  packet  losses  usually  means  that  there  is  a  deficiency  associated  with  your 
Internet  connection.  Losses  of  packets  may  reduce  upload  and  download  efficiency, 
particularly  due  to  requirements  for  retransmissions  by  applications  sensitive  to  packet 
loss,  lead  to  poor  quality  VoIP  audio,  and  pauses  in  streaming  media.  Packet  loss, 
generally  associated  with  network  congestion  and  its  inherent  packet  collisions  on 
wireless  links  or  queue-overloads,  is  a  metric  where  anything  greater  than  zero  percent 
may  be  an  issue. 

Packet  Order  is  a  measure  in  percentage  of  how  many  packets  arrived  in  order. 
Packets  do  not  necessarily  take  the  same  route  or  the  same  time  to  reach  their 
destinations.  This  results  in  packets  arriving  out  of  order,  which  causes  other  packets  to 
be  delayed  or  discarded.  Delayed  or  discarded  packets  may  cause  a  perfonnance  problem 
for  the  application,  and  as  noted  above,  may  lead  to  increased  retransmissions  which 
exacerbate  the  network  performance  issues. 
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Packet  Discards  is  a  measure  of  packets  that  arrive  too  late  to  be  used  by  the 
application.  Packet  arrivals  may  be  very  time  sensitive,  especially  with  respect  to  media- 
based  applications,  such  as  audio  or  steaming-video.  If  a  packet  arrives  too  late  the 
application  perfonnance  suffers,  and  the  packet  has  to  be  intentionally  discarded, 
effectively  wasting  the  network  resources  used  to  deliver  it. 

A  Mean  Opinion  Score  (MOS)  is  a  measure  from  1  (being  the  worst)  to  5  (being 
the  best)  as  a  rough  order  of  service  quality.  MOS  originated  from  the  phone  companies 
and  used  human  input  from  related  quality  tests.  Software  applications  have  adopted  the 
MOS  score  and  scale.  MOS  scoring  can  be  described  as  follows:  5  -  Clear,  as  if  in  a  real 
face-to-face  conversation;  4  -  Fair,  small  interference  but  sound  is  still  clear;  3  -  Not  fair, 
enough  interference  to  start  to  annoy  the  user;  2  -  Poor,  very  annoying  and  almost 
unusable;  1  -  Not  fit  for  purpose. 

Download/Upload  Speed-  more  appropriately  referred  to  as  rate  -  is  a 
measurement  of  how  fast  a  user’s  connection  can  deliver  content  to/from  their  computer. 
Note  that  this  is  generally  a  relative  measure  and  not  the  theoretical  value  for  the  link.  It 
must  also  be  specified  whether  the  value  refers  to  the  consolidated  rate  for  the  link  or  the 
effective  rate  for  individual  hosts.  For  example,  the  upload  speed  of  a  satellite  link  may 
be  1.5  Mbps;  however,  that  capacity  is  shared  among  all  users  accessing  that  link.  Thus, 
if  20  users  are  concurrently  accessing  the  link,  each  may  only  receive  75  Kbps  of  service. 
When  collecting  data  pertinent  to  upload  or  download  rates,  one  must  be  cognizant  of  the 
user  population. 

To  achieve  the  optimal  delivery  of  information  for  applications  like  VoIP,  email, 
and  on-line  interactive  programs,  the  receiving  party’s  download  rate  must  be  at  least  as 
fast  as  the  sending  party’s  upload  rate.  In  most  cases  uploading  files  is  slower  than 
downloading  files.  This  is  due  to  the  fact  that  most  Internet  connection  devices  are 
asymmetrical.  This  means  that  they  are  designed  to  provide  better  downloading  rates 
than  upload  rates.  The  reason  for  this  is  that  most  users  spend  the  majority  of  their  time 
on  the  Internet  viewing  web  pages  or  using  multimedia  files  which  involve  downloading. 
For  this  reason,  the  average  uploading  rate  is  typically  much  slower  than  the  average 
downloading  rate. 
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Round  Trip  Time  is  the  time  it  takes  for  a  packet  to  be  sent  end-to-end  between 
the  client  and  the  server  and  for  a  response  to  be  received  back  from  the  recipient.  A 
long  round  trip  time  will  dramatically  slow  connection  throughput  performance, 
particularly  for  TCP-based  applications,  and  an  erratic  round  trip  time  is  an  early 
indication  of  congestion  problems. 

2.  Baseline  Testing 

The  baseline  tests  performed  in  this  research  were  conducted  to  establish 
preliminary  network  parameters  in  ideal  settings.  These  tests  addressed  parameters  such 
as:  bandwidth,  download  and  upload  rates,  packet  losses  and  transfers,  signal  strength, 
ping,  and  jitter.  These  tests  were  perfonned  first  to  ascertain  the  best-case  values  that  are 
achievable.  These  tests  also  identify  where  major  differences  between  Femtocells  and 
Wi-Fi  exist.  The  assumptions  we  make  for  the  baseline  testing  are,  first,  that  we  have  an 
ideal  channel  that  does  not  have  any  imperfections,  interference,  or  delays;  and,  second, 
that  the  transmission  and  reception  of  the  data  takes  place  within  normal  traffic 
conditions. 

To  perform  the  baseline  tests  we  utilized  three  open-source  software  tools.  These 
included  Pingtest.net,  Speedtest.net,  and  Ping-test.net.  Pingtest.net  is  an  online 
performance-measuring  tool  that  determines  the  quality  of  the  user’s  broadband  Internet 
connection  with  respect  to  latency.  It  does  this  through  the  measuring  of  round-trip-time, 
jitter,  and  packet  loss.  This  tool  also  gives  an  overall  grade  of  the  user’s  broadband 
quality.  Speedtest.net  is  also  an  online  performance-measuring  tool  that  tests  the  users 
Internet  connection  bandwidth  with  respect  to  upload  and  download  capabilities  between 
the  assessed  client  and  a  remote  server  hosting  the  Speedtest  application.  Finally,  Ping- 
test.net  tests  the  perfonnance  of  a  user’s  Internet  connection  by  checking  how  fast  the 
user  can  download  and  upload  data.  It  accomplishes  this  by  sending  both  large  and  small 
packets  of  information  through  the  Internet  connection  and  measuring  the  speeds  of  their 
travel.  This  tool  also  addresses  latency  by  measuring  the  round-trip-time  time. 
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3.  HTTP  Web  Access  Testing 

Hypertext  Transfer  Protocol  (HTTP)  is  a  stateless,  application-layer  protocol  used 
to  transfer  data  on  the  Internet.  Web  browsers  and  servers  exchange  infonnation  in 
accordance  with  the  rules  of  HTTP.  HTTP  is  a  request/response  protocol,  which  means 
that  a  web  browser  will  initiate  a  request  to  a  server  and  the  server  in  turn  sends  a 
response,  thus  providing  an  information  “pull”  service.  HTTP  is  used  for  every  web  page 
access  and  is  used  in  every  action  involved  in  Internet  “web  surfing.” 

As  HTTP  is  the  underlying  protocol  used  by  the  World  Wide  Web,  it  is  very 
important  to  our  research.  Our  testing  addresses  HTTP  downloading  and  points  out  the 
differences  between  the  capabilities  of  Wi-Fi  and  cellular  in  terms  of  downloading  data 
from  the  Internet.  We  also  consider  the  differences  between  the  way  femtocell  and  Wi-Fi 
access  the  Internet. 

To  address  these  areas,  we  use  an  open  source  tool  entitled  “Downtester.” 
Downtester  assesses  Internet  download  speeds  from  multiple  locations  throughout  the 
world.  It  allows  the  user  to  choose  URLs  and  systematically  tests  the  download  speed  of 
each.  For  this  research,  we  chose  the  following  six  URLs:  http://www.google.com, 
http://www.facebook.com,  http://www.yahoo.com,  http://www.baidu.com, 

http://www.youtube.com,  and  http://www.cnn.com.  We  also  use  Downtester  to  measure 
the  time  to  download  files  of  varying  sizes.  We  specified  test  files  that  are  20  MB,  50MB, 
100MB,  and  200MB  in  size.  We  hold  that  these  sizes  represent  the  average  users 
download  needs.  A  20MB  file  is  considered  a  small  file,  equivalent  to  a  standard-quality 
movie  trailer  download.  A  50MB  file  is  considered  a  medium  sized  file,  equivalent  to  an 
MP3  audio  CD  download.  The  100MB  file  is  also  considered  a  medium  sized  file,  but  it 
would  equate  to  a  high-quality  MP3  audio  CD  download  or  a  2  minute  high  definition 
movie  trailer.  The  200MB  file  is  a  large  file  that  would  represent  approximately  45 
minutes  of  a  video  stream  or  a  large  operating  system  update. 

Finally,  we  also  use  HTTP  Analyzer,  which  is  a  tool  that  allows  the  user  to 
monitor,  trace,  and  analyze  HTTP  traffic  in  real-time. 
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4. 


RTP  Streaming  Testing 


To  test  streaming  capabilities  we  utilize  Real-time  Transport  Protocol  (RTP)  and 
Real-time  Transport  Control  Protocol  (RTCP).  These  protocols  provide  end-to-end 
network  transport  functions  for  applications  transmitting  real-time  data,  such  as 
interactive  audio  and  video.  Using  RTP/RTCP,  we  can  analyze  videos  of  different  sizes 
by  using  a  packet  sniffer  program.  Utilizing  a  packet  sniffer  program  and  a  streaming 
video  player,  we  can  analyze  the  stream  for  information,  such  as  sequence  errors,  jitter, 
packet  losses,  etc.  This  will  provide  a  comparison  between  the  performances  of 
Femtocells  and  Wi-Fi  for  streaming  video  content. 

To  perform  these  tests  we  use  the  packet  analyzing  freeware  program  Wireshark. 
We  use  various  size  video  files  (20MB,  50MB,  and  100MB)  to  stream  through  the 
VideoLAN  Client  (VLC)  player.  While  streaming,  we  use  Wireshark  to  capture  packets 
for  analysis. 


5.  VoIP  Tests 

Voice  Over  IP  (VoIP)  is  voice  communications  delivered  using  Internet  Protocol. 
This  means  sending  voice  infonnation  in  digital  form  in  discrete  packets  over  packet- 
switched  networks  rather  than  the  traditional  switched-circuit  sessions  used  in  Public 
Switched  Telephone  Networks  (PSTN).  In  these  tests  we  analyze  the  differences 
between  a  Wi-Fi  based  connection  and  a  Femtocell  based  connection.  It  is  generally 
accepted  that  the  better  the  upload/download  speed  the  better  the  connection  during  a 
VoIP  call.  In  the  case  of  VoIP,  jitter  and  packet  loss  are  also  factors  for  good  quality. 

To  perform  these  tests  we  use  two  open-source  online  testing  programs, 
VoIPreview.org/voipspeedtester  and  myspeed.visualware.com.  VoIPreview  is  an  online 
program  that  evaluates  the  parameters  of  a  VoIP  phone  call.  Myspeed.visualware.com  is 
an  online  program  that  provides  the  calculations  of  all  parameters  necessary  for  a 
successful  VoIP  call  and  establishes  the  value  for  an  all-encompassing  Mean  Opinion 
Score  (MOS)  this  is  shown  in  Table  6.  This  score  is  a  quality-based  score  as  introduced 
above.  These  values  are  not  always  presented  in  whole  numbers;  certain  limits  or 
thresholds  are  expressed  in  decimal  form.  The  range  of  4.0  to  4.5  is  generally  accepted 
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as  the  level  that  provides  a  quality  VoIP  call.  In  contrast,  values  below  3.5  are 
considered  unacceptable. 


1 

Impossible  to  communicate. 

2 

Very  annoying.  Nearly  impossible  to  communicate. 

3 

Annoying 

4 

Fair.  Imperfections  can  be  perceived,  but  sound  still 
clear.  This  is  the  range  for  cellular  phones. 

5 

Perfect.  Like  face-to-face  conversation  or  radio  reception. 

Table  6.  Mean  Opinion  Score  (MOS) 


D.  TESTING  CONFIGURATIONS 

In  order  to  provide  the  viable  comparison  between  Femtocells  and  Wi-Fi  in  terms 
of  their  respective  capabilities  and  perfonnance,  we  must  examine  how  they  function  in 
two  different  configurations.  Due  to  the  fact  that  Femtocell  devices  cannot  provide 
Internet  connectivity  by  themselves,  ad  hoc  networks  needed  to  be  created.  In  our 
research  two  ad  hoc  networks  were  created  for  testing  and  comparison  purposes.  The 
first  configuration  is  the  traditional  wireless  network  consisting  of  a  laptop  accessing  the 
Internet  via  a  wireless  router  (Figure6).  This  configuration  is  then  compared  to  the 
second  configuration  that  utilizes  the  Femtocell.  In  this  second  configuration,  the 
Femtocell  utilizes  the  routers  internal  Ethernet  switch  to  provide  Internet  connectivity  to 
the  Motorola  Droid  RAZR  cellular  phone  (Figure  7).  The  Droid  in  turn,  provides  a 
mobile  hotspot  that  allows  the  wireless  laptop  to  access  the  Internet. 


In  this  configuration,  tests  performed  will  be  conducted  with  a  traditional  wireless 
network  utilizing  a  wireless  laptop. 
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Figure  7.  Internet  to  wireless  router  to  femtocell  to  cellular  mobile  hotspot  to  wireless 

laptop 


In  this  configuration,  the  Femtocell  utilizes  the  wireless  router  (via  an  Ethernet 
cable)  to  provide  Internet  connectivity  to  the  Motorola  Droid  RAZR.  The  Droid  RAZR 
uses  this  to  then  provide  a  mobile  hotspot  that  is  used  by  the  wireless  laptop  to  access  the 
Internet. 

In  this  chapter  we  have  provided  the  methodology  for  the  testing  and  extensive 
experimentation  that  we  will  be  conducting  with  respect  to  these  technologies.  We 
discussed  the  technical  specifications  of  our  devices,  and  briefly  discussed  the  applicable 
testing  terms  that  will  be  used  during  our  testing.  We  have  also  described  each  of  the 
areas  where  we  will  be  testing  the  capabilities  and  performance  of  the  Femtocell  and  the 
Wi-Fi  devices.  In  our  next  chapter,  we  will  begin  our  testing.  We  will  first  establish  a 
baseline  evaluation  of  both  devices  and  utilize  this  information  to  compare  the  results  of 
further  testing  in  the  areas  of  HTTP  web  accessing,  RTP  video  streaming,  and  VoIP 
applications. 
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IV.  TESTING  AND  RESULTS  OF  TESTING 


A.  BASELINE  TESTING 

1.  Introduction 

The  baseline  tests  performed  in  this  research  were  conducted  to  establish 
preliminary  network  parameters  in  near  ideal  settings.  These  tests  address  parameters 
such  as:  bandwidth,  download  and  upload  speeds,  packet  losses  and  transfers,  signal 
strength,  round-trip-time  (RTT),  and  jitter.  These  tests  must  be  performed  first  to 
ascertain  the  best  case  values  that  are  achievable  by  each  respective  technology.  These 
tests  will  also  identify  where  the  major  differences  between  Femtocells  and  Wi-Fi  exist. 
In  this  testing,  we  must  assume  that  we  have  an  ideal  channel  that  does  not  have  any 
imperfections,  interference,  or  delays.  We  must  also  assume  that  the  transmission  and 
reception  of  the  data  takes  place  within  normal  traffic  conditions.  Testing  under  field-like 
conditions  followed  the  baseline  tests.  Performance  was  measured  using  the  three  on-line 
tools  described  in  Chapter  3.  These  included  speedtest.net  ping-test.net,  and  pingtest.net. 
The  results  of  the  tests  are  presented  below. 

2.  Baseline  Test  Evaluation 

The  baseline  tests  results  are  relatively  consistent  in  each  of  the  respective  tests. 
The  averages  are  also  what  we  expected  from  the  selected  equipment.  These  results  are 
also  in  accordance  with  what  we  would  expect  from  compliance  with  the  3GPP 
Femtocell  standards  and  IEEE  802. 1 1 . 

Wi-Fi  uses  the  same  frequency  channels  for  their  uploading  and  downloading 
operations.  It  does  this  by  utilizing  the  Carrier  Sense  Multiple  Access  with  Collision 
Avoidance  (CSMA/CA)  protocol.  This  protocol  is  designed  to  provide  fair  access  to  the 
shared  channels  so  that  all  stations  get  a  chance  to  use  a  network  (Liqiang,  2011).  After 
every  packet  is  transmitted,  all  stations  use  this  protocol  to  detennine  which  station  gets 
to  use  the  channel  next.  This  process  can  however  slow  transmission  rates.  As  shown  in 
Tables  7  through  9,  and  Figures  8  and  9,  the  results  of  our  baseline  testing  with  the  Wi-Fi 
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router  is  a  download  average  speed  of  18.87  Mb/s.  The  results  of  our  baseline  testing 
with  the  Wi-Fi  router  is  an  average  upload  speed  of  4.08  Mb/s. 


Scenario 

Ping  (ms) 

Download  Speed 
(Mb/s) 

Upload  Speed 
(Mb/s) 

Wi-Fi 

25 

18.87 

4.08 

Femtocell 

50.9 

8.06 

0.72 

Table  7.  Speedtest.net  test  averages 


Scenario 

Large 
Packet  Ping 

Small 

Packet  Ping 

Average 

Packet 

Download 

Speed 

(Mb/s) 

Upload 

Speed 

(Mb/s) 

Wi-Fi 

54.7 

52 

53.2 

14.50 

3.30 

Femtocell 

664.5 

166.7 

415.4 

9.64 

0.62 

Table  8.  Ping-test.net  test  averages 


Scenario 

Packet  Loss 

Ping 

Jitter 

Score 

Wi-Fi 

0 

24.9 

6.6 

4.39 

Femtocell 

0 

48.9 

11.3 

4.37 

Table  9.  Pingtest.net  test  averages 


Speedtest.net 


18.87 


Wi-Fi 

Femtocell 


4.08 


0.72 


Download  Speed 
(Mb/s) 


Upload  Speed  (Mb/s) 


Figure  8.  Speedtest.net  results 
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Ping-test.net 


9.64 


0.62 


Download  Speed  Upload  Speed  (Mb/s) 
(Mb/s) 


■  Wi-Fi 
Femtocell 


Figure  9.  Ping-test  results 


Femtocells  operate  differently  than  Wi-Fi  routers  in  that  they  have  different 
frequency  channels  for  both  download  and  upload  operations  (Lopez -Perez,  2008).  Since 
these  are  independent  of  one  another  you  cannot  compare  them  in  the  same  way  as  Wi-Fi 
upload  and  download  speeds.  In  Wi-Fi  you  simply  add  the  download  and  upload  speed 
averages  and  get  a  basic  speed.  For  Femtocells  you  keep  the  upload  and  download 
separate  and  measure  their  capacity  independently.  As  shown  in  Tables  7  through  9,  and 
Figures  8  and  9,  the  results  of  our  baseline  testing  show  a  download  average  speed  of 
8.06  Mb/s.  The  results  of  our  baseline  testing  show  the  upload  speed  average  of  0.72 
Mb/s  (720  Kbps). 

In  terms  of  basic  speed  the  Wi-Fi’s  downloading  speed  is  more  than  twice  as  fast 
as  the  Femtocell’s  downloading  speed.  For  uploading  operations  the  Wi-fi  is  more  than 
five  times  faster.  Downloading  and  uploading  speed  is  not  the  only  place  where  Wi-Fi 
outperforms  the  Femtocell,  packet  transmission  time  is  also  an  issue.  This  can  be  seen  by 
our  next  results  in  the  measurements  of  ping  delay  (latency)  data. 
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Ping  is  a  measurement  of  the  RTT  of  a  packet  of  infonnation  to  its  destination  and 
back.  This  includes  moving  through  the  Wi-Fi  or  Femtocell  up-link  and  down-link  as 
well  as  the  infrastructure.  If  you  were  to  assume  that  the  delay  in  ping  only  came  from 
uploading  and  downloading  values  then  the  Femtocell  ping  delay  should  be  five  times 
longer  than  the  Wi-Fi  ping  delay.  The  tests  results  however  showed  a  ratio  of  between 
seven  and  eight. 

A  possible  reason  for  this  is  that  the  Femtocell  has  a  more  complex  architecture. 
It  has  added  steps  to  the  process  of  achieving  Internet  connectivity.  It  must  first  move 
through  the  mobile  operator’s  domain  and  then  obtain  access  to  the  IP  network.  These 
steps  lead  to  a  larger  delay  in  transit  time.  This  is  an  additional  “operating  difference” 
that  needs  to  be  considered  when  evaluating  which  of  the  two  technologies  is  more 
efficient. 

Our  last  area  to  examine  is  the  jitter  results.  As  jitter  is  the  amount  of  variance  in 
successive  ping  tests  the  lower  the  value  the  better  the  connection.  In  our  tests  Wi-Fi 
jitter  averaged  6.6ms  (Table  14)  and  the  Femtocell  averaged  1 1.3ms  (Table  15).  There  is 
not  as  much  of  a  disparity  between  these  scores,  which  may  suggest  that  Femtocells  tend 
to  be  slower  in  communications  but  can  ultimately  maintain  a  stable  steady  transmission. 

Also  of  note  on  our  Pingtest.net  testing  is  the  overall  score  rating  (Tables  14  and 
15).  Pingtest.net  gives  a  Mean  Opinion  Score  (MOS),  which  is  an  indication  of  the 
overall  quality  of  the  connection.  The  MOS  ranges  from  1  to  5  with  1  being  the  worst 
and  5  being  the  best.  The  MOS  score  provided  by  this  tool  is  comprised  using  an 
algorithm  based  on  the  three  test  components  to  estimate  the  connection  quality.  Tables 
14  and  15  show  the  MOS  results  and  with  a  score  of  4.39/5  for  Wi-Fi,  and  4.37/5  for 
Femtocell  both  are  given  grades  of  A  (rated  as  excellent). 
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3. 


“Speedtest.net”  Upload/Download  Test  Results 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed 
(Mb/s) 

1 

24 

16.11 

4.08 

2 

25 

19.24 

4.20 

3 

24 

19.14 

4.15 

4 

24 

19.17 

4.20 

5 

25 

19.27 

4.17 

6 

25 

19.14 

4.15 

7 

25 

19.60 

4.17 

8 

24 

19.16 

4.20 

9 

25 

19.28 

4.16 

10 

25 

18.60 

4.17 

Average 

24.6  ms 

18.87  Mb/s 

4.08  Mb/s 

Table  10.  Wi-Fi  via  router  connection  results 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed 
(Mb/s) 

1 

55 

10.21 

0.18 

2 

41 

11.85 

0.35 

3 

55 

12.80 

1.21 

4 

65 

2.13 

0.49 

5 

38 

6.63 

1.12 

6 

45 

13.94 

1.46 

7 

45 

7.90 

1.02 

8 

55 

1.51 

0.38 

9 

55 

11.44 

0.68 

10 

55 

2.18 

0.29 

Average 

50.9  ms 

8.06  Mb/s 

0.72  Mb/s 

Table  1 1 .  Femtocell  connection  results 
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4. 


Results  of  Ping,  Jitter  and  Packet  Loss  Utilizing  “Ping-test.net”  and 
“Pingtest.net” 


Test 

Number 

Large 
Packet  Ping 
(ms) 

Small 

Packet  Ping 
(ms) 

Average 

Latency 

(ms) 

Download 

Speed 

(Mb/s) 

Upload 

Speed 

(Mb/s) 

1 

56 

56 

56 

11.38 

3.5 

2 

53 

53 

53 

17.06 

3.36 

3 

54 

51 

52 

13.96 

3.3 

4 

56 

50 

53 

13.95 

2.14 

5 

50 

54 

52 

13.71 

3.72 

6 

57 

54 

55 

16.31 

3.76 

7 

57 

50 

53 

14.13 

3.14 

8 

53 

51 

52 

14.61 

3.37 

9 

56 

51 

53 

15.21 

3.37 

10 

55 

50 

53 

14.69 

3.29 

54.7  ms 

52  ms 

53.2  ms 

14.50  Mb/s 

3.30  Mb/s 

Table  12.  Wi-Fi  connection  Ping-test.net  results  diagram 


Test 

Number 

Large 
Packet  Ping 
(ms) 

Small 

Packet  Ping 
(ms) 

Average 

Latency 

(ms) 

Download 

Speed 

(Mb/s) 

Upload 

Speed 

(Mb/s) 

1 

206 

87 

146 

5.64 

0.16 

2 

1478 

119 

798 

10.84 

0.05 

3 

646 

490 

568 

1.73 

0 

4 

1490 

254 

872 

0.84 

0 

5 

1873 

339 

1106 

4.73 

0 

6 

167 

73 

120 

13.40 

0.75 

7 

191 

87 

139 

14.93 

1.14 

8 

227 

74 

150 

14.25 

1.43 

9 

117 

74 

95 

15.36 

1.19 

10 

250 

70 

160 

14.64 

1.48 

664.5  ms 

166.7  ms 

415.4  ms 

9.64  Mb/s 

0.62  Mb/s 

Table  13.  Femtocell  connection  Ping-test.net  results  diagram 
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Test  Number 

Packet 

Loss  (%) 

Ping  (ms) 

Jitter  (ms) 

Score 
(#  out  of  5) 

1 

0 

23 

6 

4.39 

2 

0 

36 

12 

4.37 

3 

0 

21 

2 

4.39 

4 

0 

26 

10 

4.38 

5 

0 

27 

11 

4.38 

6 

0 

24 

7 

4.39 

7 

0 

27 

10 

4.38 

8 

0 

21 

2 

4.39 

9 

0 

24 

5 

4.39 

10 

0 

20 

1 

4.39 

Average 

0% 

24.9  ms 

6.6  ms 

4.39/5.0 

Table  14.  Wi-Fi  connection  Pingtest.net  results  diagram 


Test  Number 

Packet 

Loss  (%) 

Ping  (ms) 

Jitter  (ms) 

Score 
(#  out  of  5) 

1 

0 

61 

21 

4.35 

2 

0 

44 

6 

4.38 

3 

0 

51 

13 

4.36 

4 

0 

46 

6 

4.37 

5 

0 

44 

7 

4.37 

6 

0 

50 

14 

4.36 

7 

0 

51 

15 

4.36 

8 

0 

47 

10 

4.37 

9 

0 

46 

8 

4.37 

10 

0 

49 

13 

4.36 

Average 

0% 

48.9  ms 

1 1.3  ms 

4.37/5.0 

Table  15.  Femtocell  connection  Pingtest.net  results  diagram 


B.  BASELINE  TEST  IN  REALISTIC  ENVIRONMENTS 
1.  Introduction 

We  established  a  baseline  set  of  tests  and  results  in  ideal  conditions.  These  results 
will  be  useful  as  reference  points  for  successive  testing.  As  most  users  of  both  Wi-Fi  and 
Femtocell  technology  will  be  utilizing  them  in  less  than  ideal  settings  we  now  need  to 
obtain  test  results  in  realistic  environments. 
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2. 


Software  Used 


WiEye  is  a  freeware  802.11  Wi-Fi  analyzer  application  for  the  Android 
smartphone.  WiEye  is  used  for  wireless  site  surveys,  Wi-Fi  scanning,  and  wireless 
discovery.  It  displays  the  name,  Basic  Service  Set  Identification  (BSSID),  channel,  and 
frequency  of  any  access  points  within  range.  This  application  can  also  graph  this  data  to 
evaluate  Wi-Fi  congestion  for  each  available  channel.  (http://www.WiEye.net)  In  our 
research  we  will  be  utilizing  this  application  to  test  the  frequency  strength  in  our  various 
scenarios. 

OpenSignalMaps  is  a  freeware  Android  application  that  allows  the  user  to  map 
the  location  of  the  tower  to  which  the  cellular  phone  is  connected,  test  the  connection 
speed,  measure  the  exact  signal  strength,  and  provide  a  graphical  depiction  of  average 
signal  strength  over  a  period  of  time. 

Xirrus  Wi-Fi  Monitor  is  a  tool  that  provides  access  to  information  about  the  user’s 
Wi-Fi  environment  and  the  current  Wi-Fi  connection.  Xirrus  Wi-Fi  Monitor  provides 
locations  of  Wi-Fi  networks  and  their  relative  distance,  wireless  settings,  Wi-Fi  site 
survey,  signal  strength,  IP  and  MAC  addresses,  SSID,  channel,  and  security  information. 

Speedtest.net  is  an  online  perfonnance  measuring  tool  that  tests  the  user’s  Internet 
connection  bandwidth  upload  and  download  capabilities.  We  utilize  this  tool  to  measure 
our  capabilities  during  different  scenarios. 

3.  Influence  of  Obstacles 

In  less  than  ideal  or  realistic  environments,  obstacles  are  a  significant  issue.  In 
this  research  we  define  obstacles  as  those  things  that  degrade  signal  strength  and  interfere 
with  Internet  connectivity.  In  our  research  we  address  the  following  as  obstacles; 
distance,  non-supporting  walls,  and  supporting  walls.  Each  of  these  obstacles  has  a  direct 
impact  on  signal  strength  and  ultimately  the  total  amount  of  data  rate  that  is  achievable. 
For  our  ideal  baseline  testing  the  signal  strength  for  Wi-Fi  was  -58  dB(mW),  and  the 
strength  for  the  Femtocell  was  -76  dB(mW). 
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We  know  that  for  both  Wi-Fi  and  Femtocells  signal  strength  is  best  when  it 
travels  through  open  spaces  with  little  to  no  obstructions  between  wireless  transmitters 
and  wireless  receivers.  Even  when  there  are  few  obstructions  between  the  transmitter  and 
receiver,  the  overall  distance  between  them  also  affects  the  signal  strength.  Received 
signal  power  tends  to  fall  rapidly  with  distance  and  obstructions;  this  is  known  as  path 
loss.  The  capability  to  enable  low  signal  to  noise  (SNR)  operation  is  critical  to  address 
this.  Different  environments  exhibit  varying  effects  of  path  loss.  Indoor  path  loss  is 
typically  worse  than  outdoor  path  loss  due  to  greater  attenuation  of  dense  walls  and 
objects  (Chandrasekhar,  2008).  To  best  represent  these  obstacles  our  research  will  take 
both  obstacles  and  distance  into  consideration. 

4.  Testing  Scenarios 

We  have  chosen  four  different  testing  scenarios  to  best  represent  a  realistic 
environment.  These  scenarios  vary  in  distance  from  the  router  and  take  place  both 
indoors  and  outdoors.  These  testing  scenarios  also  include  the  addition  of  obstacles  in 
the  fonn  of  interior  and  exterior  walls. 

In  Scenario  1  we  test  an  indoor  area  25  feet  away  from  the  router  with  a  non¬ 
supporting  wall  between  the  router  and  the  testing  device.  Scenario  2  is  the  same 
distance  of  25  feet  but  we  have  moved  from  indoors  to  outdoors  with  the  router 
remaining  indoors;  therefore,  there  is  both  a  non-supporting  and  a  supporting  wall 
between  the  router  and  the  test  device.  In  scenario  3  we  remain  outside  with  both  a  non¬ 
supporting  and  a  supporting  wall  between  the  router  and  the  testing  device,  but  we  move 
to  50  feet  in  distance.  In  scenario  4  we  remain  outdoors  and  continue  to  have  both  non¬ 
supporting  and  supporting  walls  between  the  router  and  the  testing  device  but  move 
further  away,  to  a  distance  of  75  feet. 

Utilizing  WiEye,  OpenSignal  Maps,  and  Xirrus  Wi-Fi  monitor,  we  recorded  the 
signal  strengths  for  each  scenario: 

Baseline  Test:  -38  dB  for  Wi-Fi,  and  -56  dB  for  the  Femtocell 
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Scenario  1:  -71  dB  for  Wi-Fi  (33  dB  less  than  the  baseline),  and  -86  dB  for  the 
Femtocell  (30  dB  less  than  the  baseline) 

Scenario  2:  -74  dB  for  Wi-Fi  (36  dB  less  than  the  baseline),  and  -88  dB  for  the 
Femtocell  (32  dB  less  than  the  baseline) 

Scenario  3:  -76  dB  for  Wi-Fi  (38  dB  less  than  the  baseline),  and  -88  dB  for  the 
Femtocell  (32  dB  less  than  the  baseline) 

Scenarion  4:  -79  dB  for  Wi-Fi  (41  dB  less  than  the  baseline),  and  -94  dB  for  the 
Femtocell  (38  dB  less  than  the  baseline) 

5.  Baseline  Test  in  Realistic  Environments  Evaluation 

The  results  of  testing  in  realistic  environments  are  very  interesting.  They  show  a 
better  resistance  of  the  Femtocell  to  both  the  obstacles  and  distance.  The  signal  strength 
of  the  Femtocell  seems  to  also  be  slightly  less  attenuated  than  those  of  the  Wi-Fi. 
Overall,  throughout  the  scenarios  Wi-Fi  steadily  declined  in  perfonnance  while  the 
Femtocell  actually  increased  in  performance  until  the  final  scenario  where  they  both 
dropped  in  downloading  and  uploading  perfonnance  (Tables  16  and  17). 

In  order  to  understand  these  differences  in  performance  we  must  start  at  the 
general  canier  frequencies  of  the  two  platforms.  Wi-Fi  systems  generally  operate  at  2.4 
GHZ,  and  Femtocells  operate  at  1.8  GHZ.  Since  the  Femtocell  carrier  frequency  is  lower 
than  that  of  the  Wi-Fi  we  know  that  this  explains  why  the  Femtocell  experienced  less 
power  loss  when  compared  to  Wi-Fi  (Chandrasekhar,  2008). 

It  can  be  seen  from  scenario  1  that  a  distance  of  25  feet  and  the  insertion  of  a  non¬ 
supporting  wall  have  significant  effects  on  the  Femtocell  signal  strength.  The  baseline 
test  recorded  -56  dB,  and  the  result  of  scenario  1  was  -86  dB,  a  difference  of  30  dB.  Wi¬ 
Fi,  too,  experienced  a  significant  decrease  in  signal  strength  from  a  baseline  of  -38  dB  to 
-71dB,  a  difference  of  33  dB. 

Comparisons  of  scenarios  2  and  3  suggest  that  the  difference  between  inserting 
supporting  and  non-supporting  walls  has  a  greater  effect  on  Wi-Fi  than  on  Femtocells. 
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Wi-Fi  experienced  additional  attenuations  for  both  scenarios  where  Femtocell  results 
stayed  the  same. 

Zhen,  et  al.  (2011)  explains  that  propagation  inside  buildings  is  a  serious  problem 
for  Wireless  Local  Area  Network  (WLAN)  systems.  Zhen  goes  on  to  state  that 
Femtocells  are  a  popular  deployment  options  for  coverage  holes  and  capacity  hotspot 
areas,  and  that  “due  to  the  poor  outdoor-to-indoor  propagation  property  of  in-building 
environment,  a  dedicated  wireless  system  installed  inside  the  building  is  often  preferred 
for  providing  indoor  users  high-data-rate  services/4  Femtocells  are  cost-efficient 
techniques  for  this  application  and  results  have  shown  superior  perfonnance  of  the  Femto 
system  compared  to  other  systems  in  providing  high-data-rate  services  in  most  cases  with 
a  quality-guaranteed  scheduler,  while  the  centralized  joint  scheduling  system  gives  the 
best  performance.  The  centralized  scheme  can  also  help  improve  the  system  robustness  in 
obtaining  high  perfonnance  even  in  the  situation  where  access  points  are  placed  non¬ 
op  timally  (Zhen,  2011). 

Scenario  3  suggests  a  further  significant  decrease  in  signal  strength  when 
transmitter  and  receiver  reach  a  distance  of  75  feet  with  intervening  supporting  and  non¬ 
supporting  walls.  Noticeably,  the  decrease  in  signal  strength  is  more  remarkable  for 
Femtocells  (an  additional  6  dB)  than  for  WiFi  (an  additional  3  dB). 

In  summary,  our  realistic  environment  testing  has  shown  that  distance  and 
obstacles  weaken  WiFi  signals  in  a  very  significant  way,  much  more  than  for  Femtocells 
signals.  In  terms  of  performance  in  our  realistic  environment,  we  see  that  Wi-Fi 
download  and  uploading  speeds  tend  to  be  more  affected  by  a  less  than  ideal 
environment.  In  contrast,  the  downloading  and  uploading  performances  of  the  Femtocell 
are  only  marginally  affected  by  the  obstacles  of  a  realistic  environment.  The  Femtocell, 
therefore,  can  be  more  capable  in  certain  realistic  environments.  This  suggests  that 
Femtocells  mounted  on  maneuver  vehicles  may  provide  better  network  access  potential 
for  dismounted  personnel  inside  a  neighboring  structure  than  would  be  provided  by  a  Wi¬ 
Fi  access  point  mounted  on  the  vehicle. 
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Test 

Ping  (ms) 

Download 

Speed  (Mbps) 

Upload  Speed 
(Mbps) 

Signal  Strength 
(dBm) 

Baseline 

25 

18.87 

4.08 

-38 

Scenario  1 

26 

14.81 

4.05 

-71 

Scenario  2 

33.1 

15.04 

3.96 

-74 

Scenario  3 

27 

12.05 

3.96 

-76 

Scenario  4 

27 

9.51 

3.86 

-79 

Table  16.  Wi-Fi  averages  for  realistic  scenario  testing 


Test 

Ping  (ms) 

Download 

Speed  (Mbps) 

Upload  Speed 
(Mbps) 

Signal  Strength 
(dBm) 

Baseline 

51 

8.06 

0.72 

-56 

Scenario  1 

155 

9.70 

0.60 

-86 

Scenario  2 

467 

11.83 

0.97 

-88 

Scenario  3 

609 

10.66 

1.06 

-88 

Scenario  4 

311 

4.82 

0.53 

-94 

Table  17.  Femtocell  averages  for  realistic  scenario  testing 
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6. 


Scenario  1  Speedtest.net  Results 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed  (Mb/s) 

1 

25 

13.86 

4.05 

2 

25 

13.37 

4.17 

3 

25 

18.66 

4.17 

4 

45 

16.18 

4.12 

5 

25 

14.67 

4.12 

6 

25 

13.96 

4.19 

7 

25 

18.53 

4.11 

8 

25 

13.00 

3.86 

9 

25 

14.91 

4.17 

10 

15 

10.96 

3.53 

Average 

26  ms 

14.81  Mb/s 

4.05  Mb/s 

Table  18.  Scenario  1  Wi-Fi  connection 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed  (Mb/s) 

1 

153 

8.75 

0.57 

2 

159 

8.26 

0.58 

3 

148 

13.55 

0.58 

4 

160 

11.07 

0.56 

5 

159 

9.56 

0.53 

6 

159 

8.85 

0.62 

7 

150 

13.42 

0.49 

8 

159 

7.89 

1.04 

9 

159 

9.80 

0.58 

10 

148 

5.85 

0.42 

Average 

155  ms 

9.70  Mb/s 

0.60  Mb/s 

Table  19.  Scenario  1  Femtocell  connection 
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7. 


Scenario  2  Speedtest.net  Results 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed  (Mb/s) 

1 

35 

13.24 

4.02 

2 

25 

14.36 

4.09 

3 

25 

14.81 

4.04 

4 

46 

14.24 

3.78 

5 

65 

14.98 

3.67 

6 

15 

15.53 

3.93 

7 

55 

17.22 

4.03 

8 

25 

16.27 

3.85 

9 

15 

11.75 

4.03 

10 

25 

18.00 

4.11 

33.1  ms 

15.04  Mb/s 

3.96  Mb/s 

Table  20.  Scenario  2  Wi-Fi  connection 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed  (Mb/s) 

1 

735 

13.09 

1.13 

2 

295 

12.67 

1.15 

3 

431 

10.51 

0.77 

4 

497 

10.04 

0.76 

5 

1040 

7.88 

1.17 

6 

315 

15.38 

1.10 

7 

575 

15.53 

0.77 

8 

344 

11.97 

0.77 

9 

162 

7.55 

1.29 

10 

273 

13.70 

0.78 

467  ms 

11.83  Mb/S 

0.97  Mb/s 

Table  2 1 .  Scenario  2  Femtocell  connection 
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8. 


Scenario  3  Speedtest.net  Results 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed  (Mb/s) 

1 

15 

12.13 

3.83 

2 

45 

13.26 

4.18 

3 

25 

15.67 

3.91 

4 

35 

8.27 

3.40 

5 

25 

15.45 

4.08 

6 

25 

8.54 

4.15 

7 

25 

13.07 

4.04 

8 

25 

11.67 

3.86 

9 

25 

10.16 

4.06 

10 

25 

12.30 

4.11 

27  ms 

12.05  Mb/s 

3.96  Mb/s 

Table  22.  Scenario  3  Wi-Fi  connection 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed  (Mb/s) 

1 

431 

14.20 

1.32 

2 

508 

13.77 

1.24 

3 

579 

9.65 

0.64 

4 

174 

16.01 

0.58 

5 

920 

7.41 

1.58 

6 

923 

6.37 

1.32 

7 

581 

10.68 

0.78 

8 

522 

11.57 

1.01 

9 

658 

9.14 

1.34 

10 

268 

7.82 

0.82 

Average 

609  ms 

10.66  Mb/s 

1.06  Mb/s 

Table  23.  Scenario  3  Femtocell  connection 
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9. 


Scenario  4  Speedtest.net  Results 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed  (Mb/s) 

1 

25 

9.54 

3.97 

2 

25 

5.63 

4.07 

3 

15 

7.53 

4.06 

4 

25 

9.56 

3.93 

5 

25 

8.48 

3.85 

6 

25 

8.80 

3.91 

7 

45 

11.97 

4.06 

8 

25 

10.72 

3.53 

9 

25 

10.15 

3.33 

10 

35 

12.68 

3.90 

27  ms 

9.51  Mb/s 

3.86  Mb/s 

Table  24.  Scenario  4  Wi-Fi  connection 


Test  Number 

Ping  (ms) 

Download  Speed  (Mb/s) 

Upload  Speed  (Mb/s) 

1 

298 

4.39 

0.52 

2 

323 

2.98 

0.61 

3 

162 

3.92 

0.57 

4 

271 

5.35 

0.51 

5 

340 

3.99 

0.50 

6 

298 

4.05 

0.51 

7 

502 

6.34 

0.61 

8 

271 

5.57 

0.49 

9 

270 

5.68 

0.43 

10 

378 

5.96 

0.51 

311  ms 

4.82  Mb/s 

0.53  Mb/s 

Table  25.  Scenario  4  Femtocell  connection 


C.  ACCESSING  THE  INTERNET  THROUGH  HTTP  TESTS 
1.  Introduction 

Hypertext  Transfer  Protocol  (HTTP)  is  a  stateless  application  level  protocol  that 
is  used  to  transfer  data  on  the  Internet.  Web  browsers  and  servers  exchange  information 
in  accordance  with  the  rules  of  HTTP.  HTTP  is  a  request/response  protocol,  which 
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means  that  a  web  browser  will  initiate  a  request  to  a  server  and  the  server  in  turn  sends  a 
response.  HTTP  deals  with  every  web  page  and  is  used  in  every  action  involved  in 
Internet  “web  surfing.” 

As  HTTP  is  the  foundation  of  data  communication  for  the  World  Wide  Web 
(http://www.w3.org)  it  is  very  important  to  our  research.  Our  testing  will  therefore 
address  HTTP  downloading  and  point  out  the  differences  between  the  capabilities  of  Wi¬ 
Fi  and  Femtocell  in  terms  of  downloading  data  from  the  Internet.  We  will  also  deal  with 
the  differences  between  the  way  Femtocells  and  Wi-Fi  access  the  Internet. 

2.  Software  Used 

To  address  these  areas  we  will  use  a  freeware  tool  called  Downtester.  Downtester 
is  a  tool  that  tests  Internet  download  speeds  in  multiple  locations  throughout  the  world.  It 
allows  the  user  to  choose  URLs  and  will  systematically  test  the  download  speed  of  each. 
(http://www.nirsoft.net)  For  this  research  we  have  chosen  the  following  six  URLs: 
http://www.google.com,  http://www.facebook.com,  http://www.yahoo.com, 

http://www.baidu.com,  http://www.youtube.com,  and  http://www.cnn.com. 

We  will  also  use  the  Downtester  program  to  measure  the  time  to  download  files  of 
varying  sizes.  In  our  research  we  will  test  files  that  are  20  MB,  50MB,  100MB,  and 
200MB  in  size  (see  Tables  27  -  34).  These  sizes  were  chosen  to  represent  the  average 
users  downloading  needs,  as  stipulated  in  Chapter  3.  Finally,  we  will  also  be  using  HTTP 
Analyzer,  a  tool  that  allows  the  user  to  monitor,  trace,  and  analyze  HTTP  traffic  in  real¬ 
time  (http://www.ieinspector.com/httpanalyzer.com). 

3.  HTTP  Upload/Download  Test  Results 


File  Size 

Wi-Fi  Speed 
(Kbytes/s) 

Femtocell 

Speed 

(Kbytes/s) 

Wi-Fi  Speed 
(Mbits/s) 

Femtocell 

(Mbits/s) 

20MB 

888.30 

488.06 

7.28 

3.99 

50MB 

954.54 

524.41 

7.82 

4.29 

100MB 

873.66 

479.98 

7.16 

3.93 

200MB 

413.25 

227.03 

3.39 

1.95 

Table  26.  Average  results  comparing  Wi-Fi  and  Femtocell 
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Table  26  shows  our  HTTP  testing  downloading  rates  that  are  on  average  with  the 
results  from  our  previous  Wi-Fi  and  Femtocell  downloading  speed  comparison  testing. 
Wi-Fi’s  downloading  speed  increases  initially  and  then  with  the  larger  size  files  it  falls  a 
little  at  first  then  it  drops  significantly.  The  Femtocells  is  comparable  in  its  results.  It  too 
averages  approximately  4  Mbits/s  until  it  hits  the  largest  file  where  it  falls  to 
approximately  2  Mbits/s.  These  results  show  that  both  Femtocells  and  Wi-Fi  are 
comparable  when  accessing  files  of  various  sizes  over  the  Internet.  Only  the  biggest  files 
appear  to  slow  the  downloading  performances  of  both  Femtocells  and  Wi-Fi. 


4.  Speed  Results  for  20MB  Files 


Test  Number 

Speed  (Kbytes/s) 

Speed  (Mbits/s) 

1 

872.3 

7.15 

2 

839.6 

6.88 

3 

884.6 

7.25 

4 

566.1 

4.64 

5 

880.4 

7.21 

6 

974.8 

7.99 

7 

849.6 

6.96 

8 

951.5 

7.79 

9 

1102.3 

9.03 

10 

961.8 

7.88 

Average 

888.30  KB/Sec 

7.28  Mbps 

Table  27.  20  MB  file  Wi-Fi  connection  results 


Test  Number 

Speed  (Kbytes/s) 

Speed  (Mbits/s) 

1 

479.3 

3.92 

2 

461.3 

3.78 

3 

486.0 

3.98 

4 

311.1 

2.54 

5 

483.7 

3.96 

6 

535.6 

4.39 

7 

466.8 

3.82 

8 

522.8 

4.28 

9 

605.6 

4.96 

10 

528.4 

4.32 

Average 

488.06  KB/Sec 

3.99  Mbps 

Table  28.  20  MB  file  Femtocell  connection  results 
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5.  Speed  Results  for  50MB  Files 


Test  Number 

Speed  (Kbytes/s) 

Speed  (Mbits/s) 

1 

963.3 

7.89 

2 

974.2 

7.98 

3 

929.6 

7.62 

4 

870.1 

7.13 

5 

745.9 

6.11 

6 

1052.2 

8.62 

7 

959.6 

7.86 

8 

1003.9 

8.22 

9 

1091.6 

8.94 

10 

955.0 

7.82 

Average 

954.54  KB/Sec 

7.82  Mbps 

Table  29.  50  MB  file  Wi-Fi  connection  results 


Test  Number 

Speed  (Kbytes/s) 

Speed  (Mbits/s) 

1 

529.2 

4.33 

2 

535.2 

4.38 

3 

510.7 

4.18 

4 

478.0 

3.91 

5 

409.8 

3.35 

6 

578.1 

4.73 

7 

527.2 

4.31 

8 

551.5 

4.51 

9 

599.7 

4.91 

10 

524.7 

4.29 

Average 

524.41  KB/Sec 

4.29  Mbps 

Table  30.  50  MB  file  Femtocell  connection  results 
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6.  Speed  Results  for  100MB  Files 


Test  Number 

Speed  (Kbytes/s) 

Speed  (Mbits/s) 

1 

930.1 

7.62 

2 

947.8 

7.76 

3 

1125.3 

9.22 

4 

944.5 

7.74 

5 

1035.8 

8.49 

6 

873.6 

7.16 

7 

936.4 

7.67 

8 

941.8 

7.72 

9 

627.5 

5.14 

10 

373.8 

3.06 

Average 

873.66  KB/Sec 

7.16  Mbps 

Table  31. 

100  MB  file  Wi-Fi  connection  results 

Test  Number 

Speed  (Kbytes/s) 

Speed  (Mbits/s) 

1 

511.0 

4.18 

2 

520.7 

4.26 

3 

618.2 

5.06 

4 

518.9 

4.25 

5 

569.1 

4.66 

6 

480.0 

3.93 

7 

514.5 

4.21 

8 

517.4 

4.24 

9 

344.7 

2.82 

10 

205.3 

1.68 

Average 

479.98  KB/Sec 

3.93  Mbps 

Table  32.  100  MB  file  Femtocell  connection  results 
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7.  Speed  Results  for  200MB  Files 


Test  Number 

Speed  (Kbytes/s) 

Speed  (Mbits/s) 

1 

674.4 

5.52 

2 

412.4 

3.38 

3 

343.6 

2.81 

4 

393.1 

3.22 

5 

358.2 

2.93 

6 

318.5 

2.61 

7 

336.4 

2.76 

8 

379.6 

3.11 

9 

403.9 

3.31 

10 

512.7 

4.20 

Average 

413.25  KB/Sec 

3.39  Mbps 

Table  33. 

200  MB  file  Wi-Fi  connection  results 

Test  Number 

Speed  (Kbytes/s) 

Speed  (Mbits/s) 

1 

370.5 

3.03 

2 

226.5 

1.85 

3 

188.7 

1.54 

4 

215.9 

1.76 

5 

196.8 

1.60 

6 

175.0 

1.43 

7 

184.8 

1.51 

8 

208.5 

1.70 

9 

221.9 

1.81 

10 

281.7 

2.30 

Average 

227.03  KB/Sec 

1.85  Mbps 

Table  34.  200  MB  file  Femtocell  connection  results 


8.  Real  Time  HTTP  Monitoring 

After  establishing  our  basic  speeds  for  downloading  various  size  files  via  HTTP, 
we  now  move  on  to  the  real  time  monitoring  of  HTTP  traffic.  In  this  section  of  our 
research  we  analyze  the  process  of  HTTP  traffic.  We  will  start  with  the  components  for 
the  websites  of  the  six  chosen  websites.  These  include  Google.com,  Facebook.com, 
Youtube.com,  Yahoo.com,  CNN.com,  and  Baidu.com. 
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All  websites  are  made  up  of  similar  content.  “We  define  content  broadly  as  the 
stuff  in  your  web  site.”  (Rosenfeld  et  al,  1998)  In  general  we  view  data,  applications, 
images,  and  video  when  we  access  a  website,  which  are  all  essentially  fdes;  however,  the 
way  the  files  are  handled  and  the  sensitivity  to  loss  or  delay  is  particular  to  specific  file 
types,  based  on  the  requirements  of  the  using  applications.  As  shown  in  Figures  10  and 
11,  our  research  will  break  this  basic  content  into  percentages  of  the  total  amount  of 
components  in  each  respective  website.  We  will  use  this  as  a  reference  when  we  then 
break  down  the  six  websites  into  four  phases  of  the  HTTP  request.  The  four  phases  we 
will  cover  are  the  connect  phase,  content  request  phase,  waiting  phase,  and  receive  first  to 
last  phase.  We  will  then  compare  the  perfonnance  of  both  Wi-Fi  and  Femtocell  in 
download  these  websites. 

In  our  tests  the  connect  phase  represents  the  Transmission  Control  Protocol  (TCP) 
level  connection  time.  This  is  the  time  that  a  new  TCP  level  connection  is  established 
with  the  web  server.  The  send  first  to  last  phase  represents  the  time  required  to  send  the 
HTTP  request  message  to  the  server.  This  value  depends  on  the  amount  of  data  that  is 
sent  to  the  server.  The  wait  phase  is  the  idle  time  taken  to  receive  a  response  message 
from  the  server.  This  value  includes  network  delay  time  and  web  server  processing  time. 
Finally,  the  receive  first  to  last  phase  is  the  time  that  you  receive  the  response  message 
from  the  web  server.  This  value  depends  on  the  size  of  the  return  data  and  the  network 
bandwidth. 
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Figure  10.  Percentage  of  Wi-Fi  accessed  website  images,  text,  applications,  and  video 
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Figure  11.  Percentage  of  Femtocell  accessed  website  images,  text,  applications,  and 

video 
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Figures  10  and  11  show  the  website  profile  characteristics  based  on  the 
percentage  of  total  objects  present  in  each  of  the  listed  websites.  From  the  infonnation 
provided  by  Tables  35  and  36,  we  can  see  where  the  biggest  areas  are  and  which  affect 
performance  the  most.  Images  are  typically  the  largest  items  on  most  websites,  these  are 
followed  by  applications  and  text.  Only  one  website  visited  contained  video  files. 


Wi-Fi 

Images 

Text 

H3K5SE9I 

Video 

CNN 

48 

35 

17 

0 

Yahoo 

89 

3 

8 

0 

YouTube 

49 

28 

16 

7 

Facebook 

35 

24 

41 

0 

Baidu 

50 

13 

37 

0 

75 

12 

13 

0 

Average  % 

58% 

19% 

22% 

1% 

Table  35.  Wi-Fi  profile  characteristic  averages 


Femtocell 

Images 

Text 

Applications 

Video 

CNN 

50 

34 

16 

0 

Yahoo 

89 

3 

8 

0 

YouTube 

50 

28 

18 

4 

Facebook 

35 

24 

41 

0 

Baidu 

50 

13 

37 

0 

80 

20 

0 

0 

Average  % 

59% 

20% 

20% 

1% 

Table  36.  Femtocell  profile  characteristic  averages 


Wi-Fi 

Connect 

Send 

Wait 

Receive 

CNN 

19 

1 

48 

32 

Yahoo 

3 

1 

62 

34 

YouTube 

16 

1 

67 

16 

Facebook 

24 

1 

74 

1 

Baidu 

1 

1 

97 

1 

Google 

5 

3 

62 

30 

Average  % 

11% 

1% 

69% 

19% 

Table  37.  Wi-Fi  HTTP  breakdown 
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Femtocell 

Connect 

Send 

Wait 

Receive 

CNN 

11 

1 

40 

48 

Yahoo 

13 

1 

43 

43 

YouTube 

41 

1 

49 

9 

Facebook 

15 

1 

83 

1 

Baidu 

45 

1 

53 

1 

Google 

1 

1 

69 

29 

Average  % 

21% 

1% 

56% 

22% 

Table  38.  Femtocell  HTTP  breakdown 


Average  Beginningto  End  Times 


■  Wi-Fi 

□  Femtocell 


Figure  12.  Average  downloading  time  from  beginning  to  end  (in  seconds) 


9.  Analysis  of  HTTP  Traffic  Monitoring  Testing 

Dissecting  the  HTTP  process  in  our  testing  provides  even  better  insight  into  the 
issues  related  to  HTTP  handling  by  both  Femtocells  and  WiFi  across  a  number  of 
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different  sites  and  applications.  Our  research  shows  that  the  average  loading  time  for  a 
page  is  typically  2  seconds  for  the  Femtocell  and  slightly  less  than  that  for  Wi-Fi.  These 
speeds  are  dependent  on  the  page  that  is  accessed. 

As  shown  in  our  research  the  largest  component  of  our  selected  websites  is 
images,  at  approximately  58%  for  both  Wi-Fi  and  Femtocell.  Applications  were  the 
second  most  used  component  at  20%  for  both,  next  is  text  at  20%,  and  finally  video  was 
rarely  used  and  was  measured  at  1%. 

We  next  look  at  the  process  of  accessing  a  website.  The  four  stages  of  accessing 
a  website  are  connection,  sending,  waiting,  and  receiving.  Our  research  has  shown  that 
the  largest  percentage  of  these  four  stages  was  wait  time  (Tables  37  and  38,  and  Figure 
12).  It  was  69%  of  the  total  process  for  Wi-Fi  and  56%  for  Femtocell.  The  next  largest 
percentage  was  receiving  time  at  19%  Wi-Fi,  and  22%  for  Femtocell.  Connect  time  for 
Wi-Fi  was  11%  and  for  Femtocells  it  was  21%.  Finally,  the  smallest  percentage,  sending, 
measured  at  approximately  1%. 

Our  research  shows  us  that  the  difference  in  loading  time  is  not  necessarily 
dependent  on  the  content.  The  slight  difference  can  be  explained  by  the  Femtocells 
slower  down-load  speed  and  the  additional  delay  can  also  be  attributable  to  the  long 
mobile  operator  chain  between  the  Home  Node  (HNB)  and  global  IP  access  to  the  server 
in  the  case  of  Femtocells,  while  WiFi  can  access  directly  to  the  global  IP  network  and 
hence  to  the  server. 


D.  STREAMING  OF  FILES  THROUGH  RTP  TESTS 
1.  Introduction 

To  test  streaming  capabilities  we  utilize  Real-time  Transport  Protocol  (RTP)  and 

Real-time  Transport  Control  Protocol  (RTCP).  These  protocols  provide  end-to-end 

network  transport  functions  for  applications  transmitting  real-time  data,  such  as 

interactive  audio  and  video.  Using  RTP/RTCP  we  can  analyze  videos  of  different  sizes 

by  using  a  packet  sniffer  program.  Utilizing  a  packet  sniffer  program  we  can  analyze  the 

stream  for  infonnation  such  as  sequence  errors,  jitter,  packet  losses,  etc.  This  will 
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provide  a  comparison  between  the  perfonnances  of  Femtocells  and  Wi-Fi  while 
streaming  a  video. 

2.  Software  Used 

We  started  with  a  streaming  video  test  from  myspeed.visualware.com  that 
measures  Real  Time  Streaming  Protocol  (RTSP)  video  and  interlaced  audio  to  identify 
high  packet  jitter  and  packet  loss  that  causes  poor  quality  video.  We  then  performed  tests 
using  the  packet  analyzer,  Wireshark.  Utilizing  Wireshark  and  a  VideoLAN  Client 
(VLC)  player  we  selected  various  size  video  fdes  (20MB,  40MB,  60MB,  and  100MB)  to 
stream  (http://www.videolan.org).  While  streaming,  we  used  Wireshark  to  capture 
packets  for  analysis.  The  VLC  player  provides  the  option  to  stream  video  that  has  been 
downloaded.  We  utilized  this  option  to  stream  these  video  files  of  differing  sizes  via 
RTP  to  224.1.1.1  using  port  5004.  Wireshark  was  opened  and  started  the  capture  of 
packets  when  the  video  began  playing. 

3.  Video  and  Audio  Testing 

To  start,  we  ran  tests  on  the  quality  of  the  video  and  audio  over  IP.  This  test 
specifically  looked  at  jitter  and  packet  loss.  In  these  results  we  saw  the  variance  of  User 
Datagram  Protocol  (UDP)  RTT  over  time,  providing  insight  into  the  jitter  experienced. 
The  RTT  variance  must  be  kept  to  a  minimum  otherwise  video  quality  may  be  degraded. 
In  this  test,  the  measure  of  jitter  is  the  difference  in  time  that  each  packet  takes  to  reach 
its  destination.  In  an  ideal  case,  each  packet  sent  would  take  the  same  time  to  travel 
between  the  server  and  the  client.  In  our  tests,  we  saw  that  Wi-Fi  and  Femtocell 
measurements  in  regards  to  jitter  are  very  close  (Tables  39  and  40).  For  video,  Wi-Fi 
averages  1.54  ms  and  Femtocell  2.06  ms.  Audio  jitter  is  very  close,  with  Wi-Fi  averaging 
left  audio  channel  jitter  at  1.42  ms  and  right  audio  channel  jitter  at  1.44  ms.  Femtocells 
audio  jitter  measures  left  channel  1.18  ms  and  right  channel  1.26  ms.  In  this  case  the 
Femtocell  has  less  jitter  in  its  audio  channels.  Our  test  also  compared  the  amount  of 
packet  loss  between  Wi-Fi  and  Femtocells.  This  was  also  very  close,  with  Wi-Fi  having 
no  packet  losses  in  either  left  and  right  audio  channels  or  video.  Femtocells  had  0.16 
percent  loss  in  all  three  respective  categories  (Table  40). 
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The  final  area  addressed  in  this  test  was  packet  order.  Packet  order  is  a 
measurement  of  how  many  packets  arrive  in  order.  Packets  do  not  necessarily  take  the 
same  route  or  the  same  time  to  reach  their  destinations  and  when  they  arrive  out  of  order 
it  may  lead  to  delayed  or  even  discarded  packets.  In  our  video  test  Wi-Fi  achieved  100% 
packet  order  and  Femtocell  99.82%.  Wi-Fi  also  scored  100%  in  both  left  and  right  audio 
packet  order,  whereas  Femtocell  scored  99.82%. 


Test 

Number 

Video 

Jitter 

(ms) 

Video 

Loss 

(%) 

Video 

Packet 

Order 

(%) 

Left 

Audio 

Jitter 

(ms) 

Left 

Audio 

Loss 

(%) 

Left 

Audio 

Packet 

Order 

(%) 

Right 

Audio 

Jitter 

(ms) 

Right 

Audio 

Loss 

(%) 

Right 

Audio 

Packet 

Order 

(%) 

1 

1.5 

0 

100 

1.3 

0 

100 

1.4 

0 

100 

2 

3.2 

0 

100 

3.2 

0 

100 

3.3 

0 

100 

3 

1.6 

0 

100 

1.2 

0 

100 

1.2 

0 

100 

4 

0.7 

0 

100 

0.7 

0 

100 

0.7 

0 

100 

5 

0.7 

0 

100 

0.7 

0 

100 

0.6 

0 

100 

Average 

1.54 

ms 

0% 

100% 

1.42 

ms 

0% 

100% 

1.44 

ms 

0% 

100% 

Table  39.  Wi-Fi  audio  video  test 


Test 

Number 

Video 

Jitter 

(ms) 

Video 

Loss 

(%) 

Video 

Packet 

Order 

(%) 

Left 

Audio 

Jitter 

(ms) 

Left 

Audio 

Loss 

(%) 

Left 

Audio 

Packet 

Order 

(%) 

Right 

Audio 

Jitter 

(ms) 

Right 

Audio 

Loss 

(%) 

Right 

Audio 

Packet 

Order 

(%) 

1 

1.3 

0 

100 

1.0 

0 

100 

0.9 

0 

100 

2 

3.9 

0.8 

99.1 

2.0 

0.8 

99.1 

2.3 

0.8 

99.1 

3 

2.3 

0 

100 

1.0 

0 

100 

1.1 

0 

100 

4 

1.4 

0 

100 

0.9 

0 

100 

1.0 

0 

100 

5 

1.4 

0 

100 

1.0 

0 

100 

1.0 

0 

100 

Average 

2.06 

0.16 

99.82% 

1.18 

0.16 

99.82% 

1.26 

0.16 

99.82 

ms 

% 

ms 

% 

ms 

% 

% 

Table  40.  Femtocell  audio  video  test 


4.  Analysis  of  Video  Streaming  Through  RTP  Test  Results 

As  videos  vary  in  size  depending  on  the  quality  and  content,  we  streamed  very 


basic  videos  of  average  quality  and  with  few  special  effects  and  audio  variances  (teaching 
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and  lecture  videos).  Our  20MB  video  streamed  for  ten  minutes  and,  as  shown  for  Wi-Fi, 
there  are  no  packet  losses  or  sequence  errors.  Utilizing  the  Femtocell  for  the  same  20MB 
streaming  video  also  reflected  zero  percent  packet  loss  and  no  sequence  errors.  Both 
Wi-Fi  and  Femtocell  show  the  same  results  for  this  size  file  (Table  41).  This  is  most 
likely  due  to  the  low  amount  of  data  rate  of  the  video  streamed.  At  this  lower  level  there 
are  no  packet  losses  and  the  downlink  channel  is  not  overloaded.  This  is  very  close  to  an 
ideal  case  scenario  for  both  Wi-Fi  and  Femtocell. 

When  we  proceeded  to  the  next  set  of  files,  the  40MB  streaming  video  size.  At 
this  file  size  the  results  for  Wi-Fi  began  to  show  packet  losses  and  sequence  errors.  Wi¬ 
Fi  experienced  a  41.95%  packet  loss  and  an  average  of  8.8  sequence  errors  per  file  (Table 

42) .  While  watching  the  streaming  video  of  this  file  size  you  begin  to  see  a  slight 
degradation  in  the  quality  of  the  video.  The  packet  losses  and  sequence  errors  are  having 
an  effect  on  the  video  but  the  viewer  is  still  able  to  watch  the  video  without  too  much 
frustration.  Our  results  from  the  Femtocell  were  still  outstanding.  The  packet  losses 
were  zero  and  there  were  no  sequence  errors.  Streaming  the  video  over  Femtocell  is 
noticeably  smoother  than  the  Wi-Fi  at  this  point. 

Streaming  video  files  of  60MB  are  approximately  thirty  minutes  in  length.  Our 
Wi-Fi  streaming  tests  showed  a  continuing  decline.  The  packet  losses  increased  to 
46.09%  and  the  sequence  errors  are  just  under  8  per  file.  With  these  levels  observed 
more  distortion  and  a  few  moments  of  shuttering  (or  quick  stopping  and  jumping).  The 
Femtocell  began  to  experience  packet  loss  and  sequence  errors  streaming  this  file  size. 
The  results  from  the  Femtocell  were  28.92%  packet  loss  and  3.6  sequence  errors  (Table 

43) .  Here  we  also  observed  slight  distortion  and  image  shuddering  and  jumping. 

Videos  of  100MB  are  generally  a  higher  quality  30-40  minute  video  or  a  medium 
quality  50  minute  video.  Our  results  showed  a  significant  amount  of  packet  loss  and 
sequence  errors  for  the  Wi-Fi  and  Femtocell  at  this  size  video  file.  Wi-Fi  results  reflected 
an  average  of  81.95%  packet  loss  and  18  sequence  errors.  With  the  Wi-Fi  video  saw 
complete  stoppage  at  several  points  and  distortion.  There  are  some  instances  with 
missing  frames  that  may  be  attributed  to  the  high  levels  of  packet  loss.  For  this  file  size 
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the  Femtocell  experienced  packet  losses  of  70.79%  and  3.8  sequence  errors  (Table  44). 
The  viewing  quality  is  better  than  the  Wi-Fi,  but  still  almost  unusable. 

As  can  be  seen  by  our  research  Femtocells  outperform  Wi-Fi  in  tenns  of  RTP 
video  streaming  applications.  An  understanding  of  the  access  protocols  used  by 
Femtocells  and  WiFi  networks  provide  insight  into  the  possible  reason  for  the  difference 
in  the  perfonnance  of  these  networks  for  streaming  content.  The  3  GPP  release  6 
describes  the  High  Speed  Uplink  Packet  Access  (HSUPA)  scheme  used  by  Femtocell 
devices;  this  is  likely  the  key  to  the  Femtocells  RTP  streaming  success.  This  scheme 
tends  to  be  more  efficient  in  streaming  because  it  utilizes  the  available  spread  spectrum 
Dedicated  Physical  Data  Channel  (DPDCH),  which  transports  RTP  frames  very 
efficiently  (Hu,  et  ah,  2012).  As  streaming  operates  in  a  continuous  fashion  channels  are 
assigned  at  the  beginning  of  the  stream  and  these  same  channels  remain  used  throughout. 
The  Carrier  Sense  Multiple  Access  with  Collision  Avoidance  (CSMA/CA)  scheme  used 
by  WiFi  networks  is  a  contention=based  access  scheme  that  relies  on  collision  recovery 
mechanism  to  limit  the  effect  of  congestion  on  the  link,  but  it  does  not  eliminate 
collisions.  This  scheme  is  less  effective  for  streaming  RTP  scenarios  as  it  does  not 
dedicate  resources  to  the  specific  stream.  This  is  due  to  the  fact  that  Wi-Fi  cannot  make 
full  use  of  its  capacity  due  to  its  reliance  on  contention-based  access.  This  is  in  part 
caused  by  Wi-Fi  having  to  move  through  several  protocols  this  leads  to  a  degradation  of 
efficiency  throughout  the  streaming  process.  In  conclusion  streaming  through  RTP 
emphasizes  the  strengths  of  Femtocells  versus  Wi-Fi.  Femtocells  operate  efficiently  with 
much  less  packet  loss  and  sequence  errors  for  streaming  video  files  of  various  sizes 
(Figure  13). 
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Figure  13.  Wi-Fi  Femtocell  packet  loss 


5.  Test  Results  for  20MB  Streaming  Video 


Test 

Number 

Wi-Fi  Packet 
Loss  (%) 

Wi-Fi 

Sequence 

Errors 

Femtocell 
Packet  Loss 
(%) 

Femtocell 

Sequence 

Errors 

1 

0 

0 

0 

0 

2 

0 

0 

0 

0 

3 

0 

0 

0 

0 

4 

0 

0 

0 

0 

5 

0 

0 

0 

0 

Average 

0% 

0 

0% 

0 

Table  41.  20  MB  streaming  video  results 
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6.  Test  Results  for  40MB  Streaming  Video 


Test 

Number 

Wi-Fi  Packet 
Loss  (%) 

Wi-Fi 

Sequence 

Errors 

Femtocell 
Packet  Loss 

f%) 

Femtocell 

Sequence 

Errors 

1 

41.69 

9 

0 

0 

2 

40.02 

8 

0 

0 

3 

46.19 

9 

0 

0 

4 

41.07 

9 

0 

0 

5 

40.80 

9 

0 

0 

Average 

41.95% 

8.8 

0% 

0 

Table  42.  40  MB  streaming  video  results 


7.  Test  Results  for  60MB  Streaming  Video 


Test 

Number 

Wi-Fi  Packet 
Loss  (%) 

Wi-Fi 

Sequence 

Errors 

Femtocell 
Packet  Loss 

f%l 

Femtocell 

Sequence 

Errors 

1 

45.40 

8 

28.88 

4 

2 

40.02 

8 

27.41 

4 

3 

44.90 

7 

30.30 

3 

4 

49.30 

6 

28.55 

3 

5 

50.80 

9 

29.46 

4 

Average 

46.09% 

7.6 

28.92% 

3.6 

Table  43.  60  MB  streaming  video  results 


8.  Test  Results  for  100MB  Streaming  Video 


Test 

Number 

Wi-Fi  Packet 
Loss  (%) 

Wi-Fi 

Sequence 

Errors 

Femtocell 
Packet  Loss 
(%) 

Femtocell 

Sequence 

Errors 

1 

80.20 

19 

70.79 

4 

2 

78.50 

16 

68.28 

3 

3 

82.11 

20 

71.53 

4 

4 

83.00 

18 

70.02 

4 

5 

86.10 

17 

69.40 

4 

Average 

81.95% 

18 

70.00% 

3.8 

Table  44.  100  MB  streaming  video  results 
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E.  VOIP  TESTING 

1.  Introduction 

Voice  over  IP  (VoIP)  is  voice  delivered  using  Internet  protocol.  This  means 
sending  voice  information  in  digital  fonn  in  discrete  packets  rather  than  in  the  traditional 
circuit  switched  protocols  used  in  Public  Switched  Telephone  Networks  (PSTN).  It  is 
generally  accepted  that  the  better  the  upload/download  speed  the  better  the  connection 
during  a  VoIP  call  (Chandrasekhar,  2008).  In  the  case  of  VoIP,  jitter  and  packet  loss  are 
significant  factors  that  can  lead  to  poor  quality  or  limited  usability. 

2.  Software  Used 

To  perform  these  tests  we  used  the  online  testing  programs, 
VoIPreview.org/voipspeedtester  and  myspeed.visualware.com.  VoIPreview  is  a  program 
that  evaluates  the  parameters  of  a  VoIP  phone  call  (http://voipreview.org).  Myspeed 
provides  the  calculations  of  all  parameters  necessary  for  a  successful  VoIP  call  and 
establishes  the  evaluation  of  an  all-encompassing  Mean  Opinion  Score  (MOS)  (Table  45) 
as  described  in  Chapter  3  (http://www.myspeed.visualware.com).  The  score  depiction  is 
provided  again  for  ease  of  reference.  The  range  of  4.0  to  4.5  is  generally  accepted  as  the 
level  that  provides  a  quality  VoIP  call.  In  contrast,  any  value  below  3.5  is  considered 
unacceptable. 


Score 

Quality  of  Communication 

1 

Impossible  to  communicate. 

2 

Very  annoying.  Nearly  impossible  to  communicate. 

3 

Annoying 

4 

Fair.  Imperfections  can  be  perceived,  but  sound  still 
clear.  This  is  the  range  for  cellular  phones. 

5 

Perfect.  Like  face-to-face  conversation  or  radio  reception. 

Table  45.  Mean  Opinion  Score  (MOS) 


3.  Analysis  of  VoIP  Test  Results 

When  assessing  the  differences  between  Wi-Fi  connections  and  Femtocell 
connections  in  regards  to  VoIP  we  return  to  the  same  criteria  as  our  previous  tests;  jitter, 
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packet  loss,  and  upload/download  speed  and  consistency.  Figure  14  shows  the  VoIP 
requirements  for  jitter  and  packet  loss. 

When  considering  jitter  with  respect  to  VoIP  it  needs  to  be  less  than  5ms  to  reach 
standard  quality.  Our  testing  shows  both  Wi-Fi  and  Femtocells  jitter  scores  are  in  the 
standard  quality  range.  Wi-fi  jitter  averaged  6.8  ms  and  Femtocell  jitter  averaged  16.45 
ms.  While  the  Wi-Fi  scores  were  better,  both  are  within  acceptable  VoIP  communications 
criteria  (http :  www  .voip-info.org). 

For  VoIP  applications,  packet  loss  needs  to  be  less  than  1%  to  be  considered 
standard  quality.  Our  testing  show  packet  loss  for  Wi-Fi  is  0%,  which  is  perfect.  The 
packet  loss  for  Femtocell  was  very  close,  measuring  0.06%.  Both  Wi-Fi  and  Femtocells 
score  in  the  radio  quality  category,  which  is  perfect  for  VoIP  communications. 

Packets  are  very  time  dependent  when  it  comes  to  media  based  applications  like 
VoIP.  If  the  packet  arrives  too  late  it  is  discarded  and  there  is  a  degradation  of  quality. 
The  packet  discards  experienced  during  our  testing  was  the  first  area  where  Wi-Fi  pulls 
noticeably  ahead  of  the  Femtocell.  In  this  area  where  even  a  slight  loss  of  information 
can  noticeably  degrade  performance  Wi-Fi  had  0.04%  packet  discards  and  Femtocell  had 
1.24%. 

With  respect  to  an  overall  quality  rating,  as  indicated  by  the  assessed  MOS  score 
(see  Table  45)  Wi-Fi  score  was  3.92  and  the  Femtocell  MOS  score  was  3.54.  Both  of 
these  scores  fall  in  the  fair  category,  where  imperfections  can  be  perceived  but  sound  is 
still  clear  for  the  most  part. 

Lastly  our  test  considered  average  round  trip  time,  which  is  related  to  jitter;  the 
quicker  the  average  RTT  the  better  the  connection.  The  average  RTT  for  the  Femtocell- 
based  connection  was  longer  than  for  the  Wi-Fi.  On  average,  the  trip  time  for  Wi-Fi  was 
28ms  and  Femtocells  average  time  was  44ms.  This  is  double  the  trip  time  and  again  is 
due  to  the  longer  connection  process.  However,  44  ms  is  still  well  within  standard  for 
VoIP  service  (http:www.voip-infor.org). 
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JITTER 

PACKET  LOSS 

0  ms 

0% 

1  ms 

<-Radio  Quality-> 

2  ms 

0.2% 

3ms 

0.5% 

5  ms 

^-Standard  Quality-^ 

1% 

2% 

^-Broken  Sound-> 

5% 

<-VolP  Unsupportable^ 

50% 

100  ms 

100% 

Figure  14.  MyConnection  server  from  1999-2010  Visualware  VoIP  chart 

In  summary,  both  Wi-Fi  and  Femtocell  meet  the  requirements  for  standard  quality 
or  better  in  these  VoIP  tests.  Wi-Fi  has  a  slight,  but  only  negligible  edge,  over 
Femtocells  in  terms  of  performance  issues  for  packet  discards  and  total  averaged  round 
trip  time. 

4.  VoIP  Test  results 


Test 

Number 

Achieved 

Download 

Speed 

(Mbps) 

Achieved 

Upload 

Speed 

(Mbps) 

Download 

Service 

Consistency 

(%) 

Upload 

Service 

Consistency 

(%) 

Average 
Trip  Time 
(ms) 

1 

8.97 

4.28 

63 

97 

29 

2 

9.72 

4.25 

96 

97 

29 

3 

8.41 

3.97 

93 

92 

28 

4 

5.96 

4.12 

56 

97 

25 

5 

7.84 

3.85 

72 

96 

29 

Average 

8.18  Mbps 

4.09  Mbps 

76% 

96% 

28  ms 

Table  46.  Wi-Fi  connection  Voipreview.org/voipspeedtester  results 
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Test 

Number 

Achieved 

Download 

Speed 

(Mbps) 

Achieved 

Upload 

Speed 

(Kbps) 

Download 

Service 

Consistency 

(%) 

Upload 

Service 

Consistency 

(%) 

Average 
Trip  Time 
(ms) 

1 

8.14 

304 

90 

45 

49 

2 

6.78 

259 

56 

21 

38 

3 

7.93 

398 

69 

54 

45 

4 

9.05 

536 

86 

81 

43 

5 

8.24 

193 

90 

56 

45 

Average 

8.03  Mbps 

338  Kbps 

78.2  % 

51% 

44  ms 

Table  47.  Femtocell  connection  Voipreview.org/voipspeedtester  results 


Test 

Number 

Jitter 
User  to 
Server 
(ms) 

Jitter 
Server 
to  User 
(ms) 

Packet  Loss 
User  to 
Server  (%) 

Packet 
Loss 
Server  to 
User  (%) 

Packet 

Discards 

(%) 

MOS 

Score 

1 

3.0 

4.0 

0 

0 

0 

4.1 

2 

3.4 

13.9 

0 

0 

0 

3.9 

3 

3.8 

24.1 

0 

0 

0.2 

3.5 

4 

2.9 

3.8 

0 

0 

0 

4.1 

5 

3.9 

4.3 

0 

0 

0 

4.0 

Average 

3.4  ms 

10.2  ms 

0% 

0% 

0.04% 

3.92 

Table  48.  Wi-Fi  connection  myspeed.visualware.com  results 


Test 

Number 

Jitter 
User  to 
Server 
(ms) 

Jitter 
Server 
to  User 
(ms) 

Packet  Loss 
User  to 
Server  (%) 

Packet 
Loss 
Server  to 
User  (%) 

Packet 

Discards 

(%) 

MOS 

Score 

1 

4.0 

12.2 

0 

0 

0 

3.9 

2 

19.6 

16.6 

0 

0 

4.4 

3.7 

3 

5.5 

52.7 

0 

0 

0.4 

2.8 

4 

7.2 

24.2 

0.2 

0 

0.8 

3.6 

5 

5.8 

16.8 

0 

0.4 

0.6 

3.7 

Average 

8.4  ms 

24.5  ms 

0.04% 

0.08% 

1.24% 

3.54 

Table  49.  Femtocell  connection  myspeed.visualware.com  results 
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In  this  chapter  we  have  performed  an  in-depth  analysis  on  the  capabilities  and 
performance  of  both  Wi-Fi  and  Femtocells.  We  began  by  establishing  a  baseline  that 
pointed  out  the  benefits  of  Wi-Fi’s  better  data  rate  over  that  of  Femtocells.  However, 
when  we  perfonned  our  testing  in  less  than  ideal  conditions  we  learned  that  Wi-Fi  tends 
to  suffer  more  from  the  effects  of  obstacles  and  distance.  Our  testing  on  HTTP  web 
accessing  we  learned  that  Wi-Fi  performs  better  due  to  the  fact  that  it  was  built  and 
designed  for  this  purpose.  Femtocells  on  the  other  hand,  have  a  more  complicated 
Internet  accessing  processes  that  costs  it  in  terms  of  both  time  and  perfonnance  when 
“web  surfing.”  The  testing  performed  in  RTP  file  streaming  showed  that  Femtocells  tend 
to  operate  more  efficiently  for  larger  files  than  Wi-Fi.  Finally  our  VoIP  testing  has 
shown  that  Wi-Fi  performs  slightly  better  than  Femtocells.  We  will  now  move  on  to 
summarize  our  testing  and  provide  conclusions  as  well  as  recommendations  for  areas  for 
future  research. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS  FOR  FUTURE 

RESEARCH 


A.  CONCLUSIONS 

The  purpose  of  this  research  was  to  evaluate  the  utility  of  Femtocells  as  a  means 
of  providing  remote  access  to  smart-phone  user  where  commercial  cellular  coverage  was 
unavailable.  This  involved  comparing  the  performance  and  capabilities  provided  by 
Femtocell  enabled  networks  to  that  of  a  traditional  wireless  Wi-Fi  network.  To  do  this 
our  research  evaluated  the  performance  and  capabilities  of  these  two  technologies 
through  several  use-cases,  applications,  and  scenarios. 

Our  research  began  with  a  series  of  baseline  tests  and  assessments  of  the 
performances  of  these  two  technologies  in  near  ideal  conditions.  These  tests  were 
performed  again  in  less  than  ideal  conditions,  containing  both  obstructions  and  over 
distances  more  typical  of  conditions  experienced  by  deployed,  dismounted  force  elements 
or  first  responders.  In  ideal  conditions,  the  baseline  tests  confirmed  that  Femtocells  do 
extend  access  to  standard  cellular  systems,  but  they  offer  much  reduced  data  rates  to  a 
user  than  does  Wi-Fi  enabled  access.  This  is  due  to  the  smaller  channel  capacity  of  the 
commercial-off-the-shelf  3G  Femtocells.  Our  field  tests  provided  an  interesting  insight, 
however,  in  that  the  difference  in  raw  data  rates  was  significantly  reduced  when  the 
environment  (obstacles  and  distance)  were  entered  into  the  equation.  When  the  context  is 
in  a  realistic  field  environment,  the  Femtocells  deficiencies  are  significantly  reduced  as 
compared  to  those  of  the  WiFi  enabled  access  network  under  the  same  environmental 
conditions. 

A  potential  weakness  of  the  Femtocell  is  the  complicated  process  of  accessing  the 
Internet.  To  access  the  global  IP,  Femtocells  must  go  through  multiple  steps  in  the 
mobile  operator’s  domain  and  consequently  each  single  interaction  is  more  time 
consuming  and  complex.  This  complicated  process  adds  time  and  affects  perfonnance. 
This  is  not  the  case  for  Wi-Fi  which  connects  directly  to  the  Internet.  The  test  results 
have  shown  that  this  is  not  a  significant  time  difference  it  is  just  a  different  way  of 
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accessing  the  Internet,  as  demonstrated  by  performance  tests  in  the  area  of  Internet 
browsing,  or  “surfing,”  using  the  HTTP  protocol. 

However,  our  testing  of  web  browsing  through  HTTP  and  file  streaming  through 
RTP  indicated  very  different  results  between  Femtocells  and  Wi-Fi  hosted  access 
networks.  With  HTTP  web  browsing,  Wi-Fi  does  better  than  the  Femtocell-hosted  access 
as  it  has  the  larger  shared  channel  capacity.  It  must  be  noted,  though,  that  the  contention- 
based  access  mechanism  of  WiFi  hosted  networks  makes  it  more  susceptible  to 
congestion  (i.e.,  several  traffic  loads  associated  with  increased  numbers  of  users)  than 
does  a  Femtocell-based  access  network.  In  RTP  testing,  the  Femtocell  performed  better 
than  Wi-Fi.  The  complexity  and  time-consuming  Internet  accessing  process  for  the 
Femtocell,  that  is,  routing  all  data  traffic  through  the  cellular  provider’s  core  network  in 
order  to  access  the  Internet,  does  not  appear  to  be  as  significant  of  an  issue  with 
streaming  traffic  performance  as  it  is  with  general  web  access.  RTP  streaming  is 
essentially  a  one-way  process,  with  very  long  sequences  that  are  sensitive  to  variances  in 
packet  delays.  This  process  does  not  penalize  the  Femtocell-hosted  access  networks  as 
much  as  it  does  Wi-Fi  due  to  the  shared  channel  nature  of  the  Wi-Fi-hosted  network, 
which  is  susceptible  to  interference  from  neighboring  Wi-Fi  sources,  which  degrades  the 
respective  application  performance.  Our  tests  demonstrated  this  by  showing  that 
Femtocells  operate  correctly,  with  little  or  no  packet  loss,  for  much  larger  files  than  WiFi; 
and  even  under  more  extreme  loads,  Femtocells  packet  losses  are  half  that  of  WiFi,  until 
the  point  where  the  communications  collapse. 

Finally,  we  evaluated  the  VoIP  supporting  protocols  offered  by  both  Femtocell 
and  Wi-Fi.  Here  we  found  that  both  Wi-Fi  and  Femtocell  meet  the  requirements  for 
standard  or  better  quality  for  VoIP  operations.  Wi-Fi  has  a  slight,  but  only  negligible 
edge,  over  Femtocells  in  terms  of  performance  issues  for  packet  discards  and  total 
averaged  round  trip  time  (latency). 

In  summary,  in  baseline  testing  Wi-Fi  provides  a  better  data  rate  to  the  user  than 

does  Femtocells.  Wi-Fi,  however,  suffers  more  from  the  effects  of  obstacles  and 

distance.  In  HTTP  web  accessing,  Wi-Fi  excels  because  it  was  built  and  designed  for  this 

purpose.  Femtocells  have  complicated  Internet  accessing  processes,  requiring  an  existing 
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relationship  with  a  cellular  provider.  Further,  accessing  the  Internet  through  the  femtocell 
requires  redirection  through  that  cellular  provider  exacerbating  latency  issues.  In  RTP  file 
streaming  Femtocells  operate  more  efficiently  for  larger  files  showing  less  packet  losses 
than  Wi-Fi.  Finally  in  VoIP  testing  Wi-Fi  performs  slightly  better  than  Femtocells. 

The  use  of  smartphones,  tablets,  and  other  wireless  devices  is  becoming 
increasingly  prevalent  and  is  driving  the  need  for  innovations  in  wireless  data 
technologies  to  provide  more  capacity,  higher  speed  connections,  and  higher  quality  of 
service.  Femtocells  can  provide  a  useful  way  for  mobile  operators  to  offer  a  better  user 
experience  and  deliver  broadband  services  indoors  consistently  and  reliably  for  a 
comparable  context  of  application,  distances,  and  obstacles. 

B.  RECOMMENDATIONS  FOR  FUTURE  RESEARCH 

One  of  the  most  common  issues  with  Internet  connectivity  is  the  effect  of  multiple 
users  on  performance.  Future  research  should  explore  how  multiple  users  connected 
simultaneously  could  affect  the  behavior  and  performance  of  femtocells  and  Wi-Fi 
provided  connectivity.  Researchers  could  create  a  scenario  where  multiple  users  access 
the  network  simultaneously  and  perform  tests  that  monitors  traffic  through  all  network 
connections.  They  could  then  measure  Internet  connection  bandwidth  upload  and 
download  rates  to  compare  perfonnance  and  capabilities. 

Future  research  regarding  utilizing  the  Femtocell  in  military  applications  would 
be  beneficial.  Research  would  need  to  be  conducted  that  would  create  an  architecture  to 
incorporate  the  Femtocell  in  field  environments  and  tests  is  abilities  to  perfonn  in 
deployed  situations.  Issues  requiring  investigation  include  policy  constraints  as  well, 
particularly  in  international  contexts. 

Research  into  the  capabilities  of  utilizing  Femtocells  as  mobile  phone  range 
extenders  in  remote  areas  on  bases,  in  office  or  barracks,  and  even  on  ships  would  be 
beneficial. 

Finally,  in  this  research  we  tested  Femtocell  and  Wi-Fi  capabilities  in  both  ideal 
and  field-like  conditions  to  gain  an  understanding  of  their  respective  capabilities  and 
performance.  Future  research  could  utilize  the  same  scenarios  to  tests  impacts  to  HTTP 
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accessing,  and  streaming  via  RTP,  as  well  as  VoIP,  as  this  research  addressed  these 
impacts  for  the  ideal  environment  only. 
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